Jul 15 12:21:37.855509 ip-10-0-142-131 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 12:21:37.855525 ip-10-0-142-131 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 12:21:37.855534 ip-10-0-142-131 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 12:21:37.855873 ip-10-0-142-131 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 12:21:48.068118 ip-10-0-142-131 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 12:21:48.068133 ip-10-0-142-131 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2f541c4e6bb54f4dbb647c5037a2c2a1 -- Jul 15 12:23:19.317051 ip-10-0-142-131 systemd[1]: Starting Kubernetes Kubelet... Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.776242 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781337 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781357 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781361 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781364 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781367 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781370 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781373 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:23:19.872421 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781375 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781379 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781381 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781384 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781389 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781393 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781396 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781399 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781402 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781404 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781407 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781411 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781414 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781417 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781420 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781423 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781425 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781428 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781431 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:23:20.116906 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781433 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:23:19.860002 ip-10-0-142-131 systemd[1]: Started Kubernetes Kubelet. Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781436 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781438 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781441 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781443 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781446 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781448 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781450 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781453 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781456 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781458 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781461 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781464 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781467 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781469 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781471 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781474 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781477 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781479 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781482 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:23:20.118407 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781485 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781487 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781490 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781495 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781498 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781502 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781505 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781508 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781511 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781515 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781519 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781522 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781524 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781528 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781530 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781533 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781536 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781538 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781541 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:23:20.119310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781543 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781546 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781548 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781551 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781555 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781557 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781560 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781562 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781565 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781590 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781594 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781598 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781602 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781605 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781608 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781610 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781613 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781615 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781618 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781621 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:23:20.120094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781623 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781626 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781629 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781631 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781634 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781636 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781639 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781641 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781644 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781647 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781649 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781652 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781654 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.781657 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782368 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782375 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782378 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782381 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782384 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782387 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:23:20.120730 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782389 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782392 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782397 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782401 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782404 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782406 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782409 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782412 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782414 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782417 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782419 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782435 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782438 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782441 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782444 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782447 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782450 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782453 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782456 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782458 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:23:20.121541 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782461 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782463 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782466 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782469 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782471 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782474 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782477 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782479 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782482 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782485 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782488 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782491 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782493 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782497 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782501 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782504 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782507 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782510 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782513 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:23:20.122141 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782515 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782518 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782520 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782523 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782526 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782528 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782531 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782534 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782536 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782539 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782541 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782544 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782546 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782549 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782551 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782554 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782557 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782560 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782565 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782568 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:23:20.122755 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782571 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782575 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782578 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782580 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782583 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782586 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782589 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782592 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782594 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782597 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782599 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782603 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782605 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782608 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782610 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782613 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782615 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782618 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782620 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782623 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:23:20.123375 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782626 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782628 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782631 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782633 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782635 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782638 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782641 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782643 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782645 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782648 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782651 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782655 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782659 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.782663 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783054 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783060 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783063 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783066 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783068 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:23:20.124068 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783071 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783073 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783076 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783078 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783081 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783084 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783087 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783089 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783092 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783095 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783098 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783101 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783104 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783107 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783109 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783112 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783115 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783118 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783120 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:23:20.124617 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783123 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783125 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783128 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783131 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783135 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783137 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783140 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783144 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783146 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783149 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783151 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783154 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783156 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783159 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783162 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783164 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783167 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783169 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783171 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783174 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:23:20.125239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783177 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783179 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783181 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783184 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783187 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783189 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783192 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783194 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783197 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783199 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783202 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783204 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783207 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783209 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783211 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783214 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783217 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783220 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783222 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:23:20.125795 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783226 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783229 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783231 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783233 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783236 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783238 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783241 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783243 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783246 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783248 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783251 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783253 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783257 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783261 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783264 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783266 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783269 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783272 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783276 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783278 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:23:20.126350 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783281 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783284 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783286 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783289 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783291 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783293 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783296 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783299 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783301 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783305 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783307 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783311 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783315 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783318 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783322 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.783325 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783389 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783396 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783402 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783407 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783411 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 12:23:20.126879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783415 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783419 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783423 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783426 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783429 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783432 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783436 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783439 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783441 2559 flags.go:64] FLAG: --cgroup-root="" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783444 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783447 2559 flags.go:64] FLAG: --client-ca-file="" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783450 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783453 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783457 2559 flags.go:64] FLAG: --cluster-domain="" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783459 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783463 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783466 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783469 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783472 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783475 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783480 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783483 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783486 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783490 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 12:23:20.127505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783494 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783497 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783500 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783504 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783507 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783509 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783512 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783515 2559 flags.go:64] FLAG: --enable-server="true" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783518 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783522 2559 flags.go:64] FLAG: --event-burst="100" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783525 2559 flags.go:64] FLAG: --event-qps="50" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783528 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783531 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783534 2559 flags.go:64] FLAG: --eviction-hard="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783537 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783540 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783543 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783546 2559 flags.go:64] FLAG: --eviction-soft="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783549 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783552 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783555 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783558 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783561 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783564 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783567 2559 flags.go:64] FLAG: --feature-gates="" Jul 15 12:23:20.128140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783570 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783573 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783576 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783581 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783584 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783587 2559 flags.go:64] FLAG: --help="false" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783591 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783595 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783598 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783601 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783604 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783607 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783610 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783613 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783616 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783627 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783631 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783634 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783637 2559 flags.go:64] FLAG: --kube-reserved="" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783640 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783643 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783646 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783648 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783651 2559 flags.go:64] FLAG: --lock-file="" Jul 15 12:23:20.128839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783654 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783657 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783660 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783665 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783667 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783671 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783674 2559 flags.go:64] FLAG: --logging-format="text" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783677 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783681 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783683 2559 flags.go:64] FLAG: --manifest-url="" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783686 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783692 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783695 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783699 2559 flags.go:64] FLAG: --max-pods="110" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783705 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783708 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783711 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783714 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783717 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783720 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783723 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783730 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783733 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783736 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 12:23:20.129452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783739 2559 flags.go:64] FLAG: --pod-cidr="" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783742 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783747 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783750 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783754 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783757 2559 flags.go:64] FLAG: --port="10250" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783760 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783763 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03f1e92594826b32c" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783766 2559 flags.go:64] FLAG: --qos-reserved="" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783769 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783772 2559 flags.go:64] FLAG: --register-node="true" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783775 2559 flags.go:64] FLAG: --register-with-taints="" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783778 2559 flags.go:64] FLAG: --registry-burst="10" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783781 2559 flags.go:64] FLAG: --registry-qps="5" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783784 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783787 2559 flags.go:64] FLAG: --reserved-memory="" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783790 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783793 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783796 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783813 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783819 2559 flags.go:64] FLAG: --runonce="false" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783822 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783826 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783829 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783832 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 12:23:20.130211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783835 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783838 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783841 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783844 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783847 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783850 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783853 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783856 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783859 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783861 2559 flags.go:64] FLAG: --system-cgroups="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783864 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783869 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783872 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783875 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783878 2559 flags.go:64] FLAG: --tls-min-version="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783881 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783884 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783887 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783890 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783893 2559 flags.go:64] FLAG: --v="2" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783896 2559 flags.go:64] FLAG: --version="false" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783900 2559 flags.go:64] FLAG: --vmodule="" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783904 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.783907 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 12:23:20.131155 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784019 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784024 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784027 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784031 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784034 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784038 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784041 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784044 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784047 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784049 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784052 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784054 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784057 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784059 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784062 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784064 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784067 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784069 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784072 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784074 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:23:20.131891 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784077 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784079 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784082 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784084 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784087 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784089 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784092 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784095 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784097 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784100 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784102 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784105 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784109 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784113 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784116 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784120 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784123 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784127 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784130 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784133 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:23:20.339665 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784136 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784139 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784141 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784144 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784147 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784149 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784151 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784154 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784156 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784159 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784162 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784164 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784166 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784169 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784172 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784174 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784177 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784179 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784181 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:23:20.340551 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784184 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784186 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784189 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784191 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784194 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784197 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784199 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784201 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784206 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784209 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784213 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784216 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784218 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784221 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784223 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784226 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784228 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784231 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784233 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784236 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:23:20.341134 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784238 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784241 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784243 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784246 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784248 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784251 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784253 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784256 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784258 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784261 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784263 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784267 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784270 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784272 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784275 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784278 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784280 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784283 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784286 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:23:20.341690 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.784288 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.785071 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.795590 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.795605 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.795626 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.795632 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795689 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795693 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795697 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795700 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795703 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795706 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795709 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795711 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:23:20.342203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795714 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795717 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795720 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795723 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795725 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795728 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795731 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795733 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795736 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795739 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795742 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795748 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795750 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795753 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795756 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795758 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795761 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795764 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795766 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795770 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:23:20.342623 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795774 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795777 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795779 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795782 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795785 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795787 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795790 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795793 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795795 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795797 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795817 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795821 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795823 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795826 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795829 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795832 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795835 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795837 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795843 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:23:20.343167 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795846 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795850 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795852 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795855 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795860 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795862 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795865 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795867 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795871 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795875 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795878 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795881 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795884 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795886 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795889 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795892 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795895 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795897 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795900 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:23:20.343707 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795902 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795905 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795907 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795910 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795912 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795915 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795917 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795920 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795922 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795925 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795927 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795930 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795932 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795936 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795938 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795941 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795943 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795947 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795950 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795952 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:23:20.344237 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795955 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795957 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795960 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795963 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795965 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795968 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795970 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795973 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795976 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795979 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795981 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795984 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.795987 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.795991 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796109 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:23:20.344792 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796115 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796118 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796121 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796124 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796126 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796129 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796132 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796134 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796137 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796140 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796143 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796148 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796150 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796153 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796158 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796163 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796166 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796168 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796171 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796173 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:23:20.345208 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796176 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796179 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796182 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796185 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796187 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796189 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796192 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796194 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796197 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796200 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796203 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796205 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796208 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796210 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796213 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796215 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796218 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796220 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796223 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796225 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:23:20.345774 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796228 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796232 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796235 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796238 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796241 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796244 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796248 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796252 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796255 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796258 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796260 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796263 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796265 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796268 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796270 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796273 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796275 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796278 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796280 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796283 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:23:20.346310 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796286 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796288 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796291 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796294 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796296 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796299 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796301 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796304 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796306 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796309 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796311 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796313 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796316 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796319 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796321 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796324 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796326 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796329 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796333 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:23:20.346862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796337 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796339 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796342 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796344 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796347 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796349 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796352 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796354 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796357 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796359 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796362 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796366 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796369 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796372 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796375 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796378 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796380 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796383 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:23:20.347432 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:19.796385 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.796390 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.796504 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.799056 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.800736 2559 server.go:1019] "Starting client certificate rotation" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.800844 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.800875 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.803592 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.804701 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.814797 2559 log.go:25] "Validated CRI v1 runtime API" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.818890 2559 log.go:25] "Validated CRI v1 image API" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.820601 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 12:23:20.347978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.826120 2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a488504a-9649-49bb-95be-8e9a598ff0ab:/dev/nvme0n1p4 b3aad54e-4924-4cdd-b670-9d98c472db26:/dev/nvme0n1p3] Jul 15 12:23:20.348313 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.826136 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.832833 2559 manager.go:216] Machine: {Timestamp:2026-07-15 12:23:19.83037009 +0000 UTC m=+0.381129633 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097136 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2649106b598c780c8f6858368fc16b SystemUUID:ec264910-6b59-8c78-0c8f-6858368fc16b BootID:2f541c4e-6bb5-4f4d-bb64-7c5037a2c2a1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c7:4a:7e:61:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c7:4a:7e:61:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:d6:c5:86:84:a2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.832935 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.833011 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.836465 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.836496 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-131.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.837975 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.837989 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 12:23:20.348408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.838001 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.838688 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.840666 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.841986 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845250 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845378 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845408 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845428 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845446 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845455 2559 kubelet.go:398] "Adding apiserver pod source" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845463 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845952 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.845967 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.851398 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853694 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853721 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853913 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853925 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853931 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853936 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853944 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.853950 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.854528 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.854534 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.854540 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.854545 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.854554 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.854562 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.855506 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.855513 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.859285 2559 server.go:1268] "Started kubelet" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.860424 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.862328 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.863212 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.863244 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.863733 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-131.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.863874 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.863899 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.865460 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 12:23:20.348708 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.869175 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.868554 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-131.ec2.internal.18c2760cd3ebb7fd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-131.ec2.internal,UID:ip-10-0-142-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-131.ec2.internal,},FirstTimestamp:2026-07-15 12:23:19.859263485 +0000 UTC m=+0.410022995,LastTimestamp:2026-07-15 12:23:19.859263485 +0000 UTC m=+0.410022995,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-131.ec2.internal,}" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.871208 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.871214 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.871360 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.871994 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872003 2559 factory.go:55] Registering systemd factory Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872012 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872017 2559 factory.go:223] Registration of the systemd container factory successfully Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.871994 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872116 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872122 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872229 2559 factory.go:153] Registering CRI-O factory Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872237 2559 factory.go:223] Registration of the crio container factory successfully Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872367 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.872399 2559 factory.go:103] Registering Raw factory Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.873245 2559 manager.go:1196] Started watching for new ooms in manager Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.873749 2559 manager.go:349] Starting recovery of all containers Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.886994 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.887101 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.887528 2559 manager.go:354] Recovery completed Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.892449 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894169 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894194 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894205 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894730 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894748 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894753 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.894769 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.895351 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.895385 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.895399 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.896946 2559 policy_none.go:49] "None policy: Start" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.896959 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 12:23:20.349733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.896969 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.898925 2559 policy_none.go:47] "Start" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.903316 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xt4cb" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.903831 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-131.ec2.internal.18c2760cd6008677 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-131.ec2.internal,UID:ip-10-0-142-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-131.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-131.ec2.internal,},FirstTimestamp:2026-07-15 12:23:19.894181495 +0000 UTC m=+0.444941018,LastTimestamp:2026-07-15 12:23:19.894181495 +0000 UTC m=+0.444941018,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-131.ec2.internal,}" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.916577 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-131.ec2.internal.18c2760cd600cb21 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-131.ec2.internal,UID:ip-10-0-142-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-131.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-131.ec2.internal,},FirstTimestamp:2026-07-15 12:23:19.894199073 +0000 UTC m=+0.444958596,LastTimestamp:2026-07-15 12:23:19.894199073 +0000 UTC m=+0.444958596,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-131.ec2.internal,}" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.917255 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xt4cb" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960425 2559 manager.go:337] "Starting Device Plugin manager" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.960451 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960460 2559 server.go:87] "Starting device plugin registration server" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960667 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960679 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960786 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960877 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.960885 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.961338 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:19.961388 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:19.974105 2559 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2593 into the system container \"/system.slice\": " Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.060776 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.061774 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.061820 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.061834 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.061858 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.078646 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.078669 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-131.ec2.internal\": node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.101848 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 12:23:20.515191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.103113 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.103152 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.103173 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.103213 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.107522 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.151889 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.204238 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal"] Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.204295 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.205083 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.205108 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.205118 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.207370 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.207461 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.207483 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.211602 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.211631 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.211641 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.211602 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.211712 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.211725 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.213870 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.213891 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.214497 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.214518 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.214531 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.232311 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-131.ec2.internal\" not found" node="ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.236592 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-131.ec2.internal\" not found" node="ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.252872 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.275155 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19c3823af9989bc31c938283a398f768-config\") pod \"kube-apiserver-proxy-ip-10-0-142-131.ec2.internal\" (UID: \"19c3823af9989bc31c938283a398f768\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.515965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.275180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/909893141fa7e6fa573a50328ab8fe38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal\" (UID: \"909893141fa7e6fa573a50328ab8fe38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.275206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/909893141fa7e6fa573a50328ab8fe38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal\" (UID: \"909893141fa7e6fa573a50328ab8fe38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.355336 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.376157 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19c3823af9989bc31c938283a398f768-config\") pod \"kube-apiserver-proxy-ip-10-0-142-131.ec2.internal\" (UID: \"19c3823af9989bc31c938283a398f768\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.376183 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/909893141fa7e6fa573a50328ab8fe38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal\" (UID: \"909893141fa7e6fa573a50328ab8fe38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.376206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/909893141fa7e6fa573a50328ab8fe38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal\" (UID: \"909893141fa7e6fa573a50328ab8fe38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.376245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/909893141fa7e6fa573a50328ab8fe38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal\" (UID: \"909893141fa7e6fa573a50328ab8fe38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.376249 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/909893141fa7e6fa573a50328ab8fe38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal\" (UID: \"909893141fa7e6fa573a50328ab8fe38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.376289 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19c3823af9989bc31c938283a398f768-config\") pod \"kube-apiserver-proxy-ip-10-0-142-131.ec2.internal\" (UID: \"19c3823af9989bc31c938283a398f768\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.517585 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.456119 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.714458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.534442 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.714458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.540522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:20.714458 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.556898 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.714458 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.660911 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.908221 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.761591 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:20.908221 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.761766 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:23:20.908221 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.800794 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 12:23:20.908221 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.800946 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 12:23:20.908221 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.800950 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 12:23:20.908221 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.862242 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.921699 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 12:18:20 +0000 UTC" deadline="2027-12-14 23:26:34.36417169 +0000 UTC" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:20.921728 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12419h3m13.442449276s" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:20.963043 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.051330 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.052895 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.052918 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:23:21.062031 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.052928 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:23:21.064124 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.063307 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:21.066823 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.066547 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:23:21.181129 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.164153 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:21.355504 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:21.223501 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c3823af9989bc31c938283a398f768.slice/crio-6566a345a2457586fca5d2aa7c27d55c49f6a329b1bb1656ee703b1f661c47b4 WatchSource:0}: Error finding container 6566a345a2457586fca5d2aa7c27d55c49f6a329b1bb1656ee703b1f661c47b4: Status 404 returned error can't find the container with id 6566a345a2457586fca5d2aa7c27d55c49f6a329b1bb1656ee703b1f661c47b4 Jul 15 12:23:21.355504 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:21.224026 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909893141fa7e6fa573a50328ab8fe38.slice/crio-4683e5132505b870ee882b6fc88a153fb2b406e9fc91047aa6d815e6993f3188 WatchSource:0}: Error finding container 4683e5132505b870ee882b6fc88a153fb2b406e9fc91047aa6d815e6993f3188: Status 404 returned error can't find the container with id 4683e5132505b870ee882b6fc88a153fb2b406e9fc91047aa6d815e6993f3188 Jul 15 12:23:21.355504 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.229315 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:23:21.355504 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.265180 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-131.ec2.internal\" not found" Jul 15 12:23:21.355504 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.344329 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:23:21.581537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.372351 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" Jul 15 12:23:21.581537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.388675 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 12:23:21.581537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.389786 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" Jul 15 12:23:21.581537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.421031 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.847219 2559 apiserver.go:52] "Watching apiserver" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.857053 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.857464 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-xksn9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal","openshift-multus/network-metrics-daemon-dcp89","openshift-network-diagnostics/network-check-target-mbn5n","openshift-network-operator/iptables-alerter-5bf7h","openshift-ovn-kubernetes/ovnkube-node-swsfr","kube-system/konnectivity-agent-m2qqp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn","openshift-cluster-node-tuning-operator/tuned-lz2rl","openshift-dns/node-resolver-nr9tq","openshift-multus/multus-9f2dk","openshift-multus/multus-additional-cni-plugins-qfkst","kube-system/global-pull-secret-syncer-5qqkj","kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal"] Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.860212 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.862512 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.863746 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.863752 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.863764 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bbq2d\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.865222 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.865293 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.865710 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.865733 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pgv42\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.866044 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.867641 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.867713 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.870050 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.872554 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.872671 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.873682 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.873757 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.873794 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.874021 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.874151 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m6j6l\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.874520 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.875535 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.903033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.876130 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.877935 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.881599 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z4tt6\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.881599 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.881626 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.881907 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.882382 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.882974 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vchks\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883003 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883160 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883209 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-hostroot\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883232 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjx2d\" (UniqueName: \"kubernetes.io/projected/c79cb6fd-6a97-4b80-8f34-5106f3096787-kube-api-access-jjx2d\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883250 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-os-release\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysconfig\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883301 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883320 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysctl-conf\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883346 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ssjzw\"" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883357 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-lib-modules\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.904776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883390 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-netns\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883413 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhdj6\" (UniqueName: \"kubernetes.io/projected/09106fe3-c171-4e91-b958-ba44ff83852e-kube-api-access-dhdj6\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883450 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-k8s-cni-cncf-io\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883484 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-daemon-config\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-etc-kubernetes\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883541 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/489c2f9f-1652-4042-9366-dbb76fac0840-hosts-file\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/489c2f9f-1652-4042-9366-dbb76fac0840-tmp-dir\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883588 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-systemd\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883612 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-cni-multus\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-kubelet\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-var-lib-kubelet\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883687 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmdnf\" (UniqueName: \"kubernetes.io/projected/489c2f9f-1652-4042-9366-dbb76fac0840-kube-api-access-pmdnf\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883742 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-modprobe-d\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.906138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883771 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-kubernetes\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883822 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysctl-d\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883842 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-conf-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883858 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-system-cni-dir\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883877 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cdc00c95-fb22-4c7d-858d-f6aac54291cc-konnectivity-ca\") pod \"konnectivity-agent-m2qqp\" (UID: \"cdc00c95-fb22-4c7d-858d-f6aac54291cc\") " pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cni-binary-copy\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883924 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-run\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883950 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-tmp\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsmmj\" (UniqueName: \"kubernetes.io/projected/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-kube-api-access-zsmmj\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.883984 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-system-cni-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884009 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-cni-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884039 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-cni-bin\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-multus-certs\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884098 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-sys\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.907055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-host\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884155 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cnibin\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884175 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vvh4\" (UniqueName: \"kubernetes.io/projected/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-kube-api-access-6vvh4\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-tuned\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884243 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-socket-dir-parent\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884267 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884290 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884307 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cdc00c95-fb22-4c7d-858d-f6aac54291cc-agent-certs\") pod \"konnectivity-agent-m2qqp\" (UID: \"cdc00c95-fb22-4c7d-858d-f6aac54291cc\") " pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884321 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-cnibin\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-os-release\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.884362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c79cb6fd-6a97-4b80-8f34-5106f3096787-cni-binary-copy\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.890046 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.890545 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.893087 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.894536 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.894554 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 12:23:21.908296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.895122 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903286 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8f6cc\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903406 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903497 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903594 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903595 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903729 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q8cjk\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.903893 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 12:23:21.909537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.906263 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:21.910234 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.909578 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.913856 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.913869 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.913986 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-knk7r\"" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.914204 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.926047 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dxgj5" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.940144 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dxgj5" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.973446 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-sys-fs\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985396 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-run\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985414 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-system-cni-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985430 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-sys\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985499 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-sys\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985539 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-host\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985558 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-system-cni-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985575 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-run\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vvh4\" (UniqueName: \"kubernetes.io/projected/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-kube-api-access-6vvh4\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985608 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-host\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.010263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985633 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-tuned\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985659 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-socket-dir-parent\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985685 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-hostroot\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985710 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985789 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-socket-dir-parent\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985816 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-hostroot\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cdc00c95-fb22-4c7d-858d-f6aac54291cc-agent-certs\") pod \"konnectivity-agent-m2qqp\" (UID: \"cdc00c95-fb22-4c7d-858d-f6aac54291cc\") " pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985923 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-cnibin\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985932 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-os-release\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985970 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-cnibin\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.985979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjx2d\" (UniqueName: \"kubernetes.io/projected/c79cb6fd-6a97-4b80-8f34-5106f3096787-kube-api-access-jjx2d\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986004 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-registration-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjwml\" (UniqueName: \"kubernetes.io/projected/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-kube-api-access-mjwml\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986027 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-os-release\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.012700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986144 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-os-release\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986150 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67367022-d0b3-4781-a0f2-5e85dbb73156-iptables-alerter-script\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhdj6\" (UniqueName: \"kubernetes.io/projected/09106fe3-c171-4e91-b958-ba44ff83852e-kube-api-access-dhdj6\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.986209 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-daemon-config\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986240 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:21.986287 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:22.486266354 +0000 UTC m=+3.037025870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986332 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-systemd-units\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-run-netns\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986410 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b314c57b-6e4c-439a-80c3-caba783d364d-ovn-node-metrics-cert\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986463 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/489c2f9f-1652-4042-9366-dbb76fac0840-hosts-file\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.013575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986493 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-os-release\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-kubelet\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986513 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/489c2f9f-1652-4042-9366-dbb76fac0840-hosts-file\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986538 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-kubelet\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986565 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-systemd\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986592 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-device-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-var-lib-kubelet\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986645 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986681 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-modprobe-d\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986703 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-conf-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986750 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-system-cni-dir\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cdc00c95-fb22-4c7d-858d-f6aac54291cc-konnectivity-ca\") pod \"konnectivity-agent-m2qqp\" (UID: \"cdc00c95-fb22-4c7d-858d-f6aac54291cc\") " pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986778 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-modprobe-d\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.015410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986792 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-cni-netd\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986840 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-conf-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986680 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-var-lib-kubelet\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986868 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-system-cni-dir\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cni-binary-copy\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986812 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-daemon-config\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986914 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-tmp\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986947 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsmmj\" (UniqueName: \"kubernetes.io/projected/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-kube-api-access-zsmmj\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.986979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-cni-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-cni-bin\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987041 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-multus-certs\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987072 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-ovnkube-script-lib\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cnibin\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.016043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987180 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987181 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-socket-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987301 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cni-binary-copy\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987351 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-cnibin\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987357 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cdc00c95-fb22-4c7d-858d-f6aac54291cc-konnectivity-ca\") pod \"konnectivity-agent-m2qqp\" (UID: \"cdc00c95-fb22-4c7d-858d-f6aac54291cc\") " pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987388 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-cni-bin\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987403 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741e3629-2b48-4694-a392-189d31991a8f-dbus\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987417 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-multus-cni-dir\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987429 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-multus-certs\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987465 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-cni-bin\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987489 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90572c3f-6cb5-4c41-8b12-222f3041b25d-host\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987580 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987641 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c79cb6fd-6a97-4b80-8f34-5106f3096787-cni-binary-copy\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-var-lib-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.018862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987732 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysconfig\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987759 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysctl-conf\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987789 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-lib-modules\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-netns\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-etc-selinux\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987946 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-slash\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.987977 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-ovnkube-config\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988012 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-env-overrides\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988052 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-k8s-cni-cncf-io\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988089 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-etc-kubernetes\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988128 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741e3629-2b48-4694-a392-189d31991a8f-kubelet-config\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988164 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-kubelet\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-etc-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988236 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.019549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988274 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h62s\" (UniqueName: \"kubernetes.io/projected/b314c57b-6e4c-439a-80c3-caba783d364d-kube-api-access-6h62s\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988389 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/489c2f9f-1652-4042-9366-dbb76fac0840-tmp-dir\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988421 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-systemd\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988455 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-cni-multus\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988494 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzxbb\" (UniqueName: \"kubernetes.io/projected/67367022-d0b3-4781-a0f2-5e85dbb73156-kube-api-access-mzxbb\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988534 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-ovn\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988559 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-lib-modules\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988568 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-node-log\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988591 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c79cb6fd-6a97-4b80-8f34-5106f3096787-cni-binary-copy\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988610 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmdnf\" (UniqueName: \"kubernetes.io/projected/489c2f9f-1652-4042-9366-dbb76fac0840-kube-api-access-pmdnf\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988642 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-kubernetes\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988650 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysconfig\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988659 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-etc-kubernetes\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988666 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysctl-d\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988459 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysctl-conf\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.022799 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988706 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-var-lib-cni-multus\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988750 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-sysctl-d\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988776 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-log-socket\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988785 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-systemd\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988789 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-k8s-cni-cncf-io\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988803 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/90572c3f-6cb5-4c41-8b12-222f3041b25d-serviceca\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988817 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c79cb6fd-6a97-4b80-8f34-5106f3096787-host-run-netns\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988829 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67367022-d0b3-4781-a0f2-5e85dbb73156-host-slash\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988873 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-run-ovn-kubernetes\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.988920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46bc\" (UniqueName: \"kubernetes.io/projected/90572c3f-6cb5-4c41-8b12-222f3041b25d-kube-api-access-r46bc\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.989019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-kubernetes\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.989164 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/489c2f9f-1652-4042-9366-dbb76fac0840-tmp-dir\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.990178 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-tmp\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.990334 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cdc00c95-fb22-4c7d-858d-f6aac54291cc-agent-certs\") pod \"konnectivity-agent-m2qqp\" (UID: \"cdc00c95-fb22-4c7d-858d-f6aac54291cc\") " pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:22.023452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:21.991106 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-etc-tuned\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.026764 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.026346 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjx2d\" (UniqueName: \"kubernetes.io/projected/c79cb6fd-6a97-4b80-8f34-5106f3096787-kube-api-access-jjx2d\") pod \"multus-9f2dk\" (UID: \"c79cb6fd-6a97-4b80-8f34-5106f3096787\") " pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.039106 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.039128 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.039144 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.039228 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:22.539208515 +0000 UTC m=+3.089968029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.041319 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vvh4\" (UniqueName: \"kubernetes.io/projected/7848a9fc-b82c-4f02-9bcd-7e43fa24fbae-kube-api-access-6vvh4\") pod \"multus-additional-cni-plugins-qfkst\" (UID: \"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae\") " pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.041719 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmdnf\" (UniqueName: \"kubernetes.io/projected/489c2f9f-1652-4042-9366-dbb76fac0840-kube-api-access-pmdnf\") pod \"node-resolver-nr9tq\" (UID: \"489c2f9f-1652-4042-9366-dbb76fac0840\") " pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.041736 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsmmj\" (UniqueName: \"kubernetes.io/projected/c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf-kube-api-access-zsmmj\") pod \"tuned-lz2rl\" (UID: \"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf\") " pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.042813 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhdj6\" (UniqueName: \"kubernetes.io/projected/09106fe3-c171-4e91-b958-ba44ff83852e-kube-api-access-dhdj6\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.089995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-node-log\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-log-socket\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/90572c3f-6cb5-4c41-8b12-222f3041b25d-serviceca\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090169 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67367022-d0b3-4781-a0f2-5e85dbb73156-host-slash\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-run-ovn-kubernetes\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.114456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46bc\" (UniqueName: \"kubernetes.io/projected/90572c3f-6cb5-4c41-8b12-222f3041b25d-kube-api-access-r46bc\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090286 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-sys-fs\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-registration-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090372 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjwml\" (UniqueName: \"kubernetes.io/projected/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-kube-api-access-mjwml\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090421 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67367022-d0b3-4781-a0f2-5e85dbb73156-iptables-alerter-script\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090542 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-systemd-units\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-run-netns\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b314c57b-6e4c-439a-80c3-caba783d364d-ovn-node-metrics-cert\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090629 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-systemd\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090654 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-device-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-cni-netd\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.115650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090806 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-ovnkube-script-lib\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090833 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/90572c3f-6cb5-4c41-8b12-222f3041b25d-serviceca\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090848 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-socket-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090907 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741e3629-2b48-4694-a392-189d31991a8f-dbus\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090934 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-node-log\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090965 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67367022-d0b3-4781-a0f2-5e85dbb73156-host-slash\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090983 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-device-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090999 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091016 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-run-ovn-kubernetes\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091106 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091165 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741e3629-2b48-4694-a392-189d31991a8f-dbus\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091189 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-socket-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091213 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-run-netns\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091285 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-registration-dir\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.116437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-cni-netd\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091447 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-sys-fs\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.090377 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-log-socket\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-cni-bin\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091505 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-systemd\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90572c3f-6cb5-4c41-8b12-222f3041b25d-host\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-var-lib-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091583 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-etc-selinux\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091612 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-slash\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091631 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-ovnkube-config\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091657 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-env-overrides\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741e3629-2b48-4694-a392-189d31991a8f-kubelet-config\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091709 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-kubelet\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-etc-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.192185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091776 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091803 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h62s\" (UniqueName: \"kubernetes.io/projected/b314c57b-6e4c-439a-80c3-caba783d364d-kube-api-access-6h62s\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091808 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-etc-selinux\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091830 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzxbb\" (UniqueName: \"kubernetes.io/projected/67367022-d0b3-4781-a0f2-5e85dbb73156-kube-api-access-mzxbb\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091834 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67367022-d0b3-4781-a0f2-5e85dbb73156-iptables-alerter-script\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091848 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-cni-bin\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091855 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-ovn\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091942 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-kubelet\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-host-slash\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.091996 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-etc-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90572c3f-6cb5-4c41-8b12-222f3041b25d-host\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092043 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741e3629-2b48-4694-a392-189d31991a8f-kubelet-config\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092063 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-systemd-units\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092084 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-ovn\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092096 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-var-lib-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.193506 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.092161 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.092224 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:23:22.592198742 +0000 UTC m=+3.142958269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092249 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b314c57b-6e4c-439a-80c3-caba783d364d-run-openvswitch\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092500 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-env-overrides\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.092866 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-ovnkube-script-lib\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.093017 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b314c57b-6e4c-439a-80c3-caba783d364d-ovnkube-config\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.096658 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b314c57b-6e4c-439a-80c3-caba783d364d-ovn-node-metrics-cert\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.117205 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h62s\" (UniqueName: \"kubernetes.io/projected/b314c57b-6e4c-439a-80c3-caba783d364d-kube-api-access-6h62s\") pod \"ovnkube-node-swsfr\" (UID: \"b314c57b-6e4c-439a-80c3-caba783d364d\") " pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.117668 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46bc\" (UniqueName: \"kubernetes.io/projected/90572c3f-6cb5-4c41-8b12-222f3041b25d-kube-api-access-r46bc\") pod \"node-ca-xksn9\" (UID: \"90572c3f-6cb5-4c41-8b12-222f3041b25d\") " pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.117714 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzxbb\" (UniqueName: \"kubernetes.io/projected/67367022-d0b3-4781-a0f2-5e85dbb73156-kube-api-access-mzxbb\") pod \"iptables-alerter-5bf7h\" (UID: \"67367022-d0b3-4781-a0f2-5e85dbb73156\") " pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.118276 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" event={"ID":"909893141fa7e6fa573a50328ab8fe38","Type":"ContainerStarted","Data":"4683e5132505b870ee882b6fc88a153fb2b406e9fc91047aa6d815e6993f3188"} Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.119323 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjwml\" (UniqueName: \"kubernetes.io/projected/ba71e6e6-8a29-4ffd-a64e-28d9f7509cff-kube-api-access-mjwml\") pod \"aws-ebs-csi-driver-node-7prtn\" (UID: \"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.121667 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" event={"ID":"19c3823af9989bc31c938283a398f768","Type":"ContainerStarted","Data":"6566a345a2457586fca5d2aa7c27d55c49f6a329b1bb1656ee703b1f661c47b4"} Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.172615 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:22.194208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.182382 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nr9tq" Jul 15 12:23:22.195560 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.186531 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc00c95_fb22_4c7d_858d_f6aac54291cc.slice/crio-c0924f8d1267b70690824a9cec2b332b2e08e4cc185edd5af19fff595cdf14e7 WatchSource:0}: Error finding container c0924f8d1267b70690824a9cec2b332b2e08e4cc185edd5af19fff595cdf14e7: Status 404 returned error can't find the container with id c0924f8d1267b70690824a9cec2b332b2e08e4cc185edd5af19fff595cdf14e7 Jul 15 12:23:22.196701 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.195769 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qfkst" Jul 15 12:23:22.197543 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.197524 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9f2dk" Jul 15 12:23:22.204966 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.204929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" Jul 15 12:23:22.216885 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.216857 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5bf7h" Jul 15 12:23:22.219266 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.219228 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489c2f9f_1652_4042_9366_dbb76fac0840.slice/crio-f1eff52ab65a0a6455160d506a2057180249d5377b11af61d1403c259f873824 WatchSource:0}: Error finding container f1eff52ab65a0a6455160d506a2057180249d5377b11af61d1403c259f873824: Status 404 returned error can't find the container with id f1eff52ab65a0a6455160d506a2057180249d5377b11af61d1403c259f873824 Jul 15 12:23:22.221297 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.221270 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7848a9fc_b82c_4f02_9bcd_7e43fa24fbae.slice/crio-3126449d8edc7a5be0ef0a3f0464759fd65dcaeada609ab08a365ca1bfe7259f WatchSource:0}: Error finding container 3126449d8edc7a5be0ef0a3f0464759fd65dcaeada609ab08a365ca1bfe7259f: Status 404 returned error can't find the container with id 3126449d8edc7a5be0ef0a3f0464759fd65dcaeada609ab08a365ca1bfe7259f Jul 15 12:23:22.228574 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.228547 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79cb6fd_6a97_4b80_8f34_5106f3096787.slice/crio-51f7eb7bc1ea6904c79ea7a26450640c496a8e27578c217dd9e1026cd020bae1 WatchSource:0}: Error finding container 51f7eb7bc1ea6904c79ea7a26450640c496a8e27578c217dd9e1026cd020bae1: Status 404 returned error can't find the container with id 51f7eb7bc1ea6904c79ea7a26450640c496a8e27578c217dd9e1026cd020bae1 Jul 15 12:23:22.228777 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.228759 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:22.230927 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.230829 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a8cf64_9dce_4bcf_aba1_b72ddde9cfaf.slice/crio-2d184d0d98ff300eb5444ad9937cbaa1edd2e32b63010ab89879c018d432ba50 WatchSource:0}: Error finding container 2d184d0d98ff300eb5444ad9937cbaa1edd2e32b63010ab89879c018d432ba50: Status 404 returned error can't find the container with id 2d184d0d98ff300eb5444ad9937cbaa1edd2e32b63010ab89879c018d432ba50 Jul 15 12:23:22.235741 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.235576 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xksn9" Jul 15 12:23:22.244158 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.244133 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" Jul 15 12:23:22.256140 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.256112 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90572c3f_6cb5_4c41_8b12_222f3041b25d.slice/crio-b3a27b62460acb9799c7b093603a5cf2e58e96c635cb1a54a42ca99e2e4378bb WatchSource:0}: Error finding container b3a27b62460acb9799c7b093603a5cf2e58e96c635cb1a54a42ca99e2e4378bb: Status 404 returned error can't find the container with id b3a27b62460acb9799c7b093603a5cf2e58e96c635cb1a54a42ca99e2e4378bb Jul 15 12:23:22.257599 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.257554 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb314c57b_6e4c_439a_80c3_caba783d364d.slice/crio-82491e5db07cfe8a42b0f8b6101501ccaff9357efe4b436dcfb7c6a84d9d7bb4 WatchSource:0}: Error finding container 82491e5db07cfe8a42b0f8b6101501ccaff9357efe4b436dcfb7c6a84d9d7bb4: Status 404 returned error can't find the container with id 82491e5db07cfe8a42b0f8b6101501ccaff9357efe4b436dcfb7c6a84d9d7bb4 Jul 15 12:23:22.267786 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:22.267754 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba71e6e6_8a29_4ffd_a64e_28d9f7509cff.slice/crio-3d9468c47e22e6f0582abd294648e12b1c189bab9f52c5c875a2e9dd46d8406b WatchSource:0}: Error finding container 3d9468c47e22e6f0582abd294648e12b1c189bab9f52c5c875a2e9dd46d8406b: Status 404 returned error can't find the container with id 3d9468c47e22e6f0582abd294648e12b1c189bab9f52c5c875a2e9dd46d8406b Jul 15 12:23:22.316918 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.316887 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:23:22.498395 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.498363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:22.498588 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.498489 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:22.498588 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.498566 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:23.498543935 +0000 UTC m=+4.049303454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:22.599442 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.599407 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:22.599622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.599480 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:22.599622 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.599577 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:22.599622 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.599601 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:22.599622 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.599621 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:22.599842 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.599637 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:22.599842 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.599645 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:23:23.599629345 +0000 UTC m=+4.150388856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:22.599842 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:22.599688 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:23.599671807 +0000 UTC m=+4.150431330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:22.826468 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.826391 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:23:22.943515 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.943438 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:18:22 +0000 UTC" deadline="2028-02-19 23:11:59.145676808 +0000 UTC" Jul 15 12:23:22.944275 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:22.943556 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14026h48m36.202169181s" Jul 15 12:23:23.103455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.103371 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:23.103455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.103411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:23.103455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.103417 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:23.103718 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.103502 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:23.103718 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.103605 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:23.103817 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.103742 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:23.124285 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.124252 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m2qqp" event={"ID":"cdc00c95-fb22-4c7d-858d-f6aac54291cc","Type":"ContainerStarted","Data":"c0924f8d1267b70690824a9cec2b332b2e08e4cc185edd5af19fff595cdf14e7"} Jul 15 12:23:23.126050 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.126024 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5bf7h" event={"ID":"67367022-d0b3-4781-a0f2-5e85dbb73156","Type":"ContainerStarted","Data":"67f3da3c02881d166fddbf272b0975e4b42515975339c334697fada16cb785de"} Jul 15 12:23:23.127481 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.127456 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" event={"ID":"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf","Type":"ContainerStarted","Data":"2d184d0d98ff300eb5444ad9937cbaa1edd2e32b63010ab89879c018d432ba50"} Jul 15 12:23:23.129491 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.129412 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9f2dk" event={"ID":"c79cb6fd-6a97-4b80-8f34-5106f3096787","Type":"ContainerStarted","Data":"51f7eb7bc1ea6904c79ea7a26450640c496a8e27578c217dd9e1026cd020bae1"} Jul 15 12:23:23.132540 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.132435 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" event={"ID":"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff","Type":"ContainerStarted","Data":"3d9468c47e22e6f0582abd294648e12b1c189bab9f52c5c875a2e9dd46d8406b"} Jul 15 12:23:23.145612 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.145493 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"82491e5db07cfe8a42b0f8b6101501ccaff9357efe4b436dcfb7c6a84d9d7bb4"} Jul 15 12:23:23.148005 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.147978 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xksn9" event={"ID":"90572c3f-6cb5-4c41-8b12-222f3041b25d","Type":"ContainerStarted","Data":"b3a27b62460acb9799c7b093603a5cf2e58e96c635cb1a54a42ca99e2e4378bb"} Jul 15 12:23:23.149888 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.149856 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerStarted","Data":"3126449d8edc7a5be0ef0a3f0464759fd65dcaeada609ab08a365ca1bfe7259f"} Jul 15 12:23:23.151730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.151704 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nr9tq" event={"ID":"489c2f9f-1652-4042-9366-dbb76fac0840","Type":"ContainerStarted","Data":"f1eff52ab65a0a6455160d506a2057180249d5377b11af61d1403c259f873824"} Jul 15 12:23:23.507264 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.506585 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:23.507264 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.506776 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:23.507264 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.506862 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:25.506824163 +0000 UTC m=+6.057583680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.607857 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.607940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.608081 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.608142 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:23:25.608123947 +0000 UTC m=+6.158883462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.608587 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.608613 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.608628 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:23.608717 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:23.608679 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:25.608661205 +0000 UTC m=+6.159420737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:23.944685 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.944568 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:18:22 +0000 UTC" deadline="2028-03-20 18:33:57.837240939 +0000 UTC" Jul 15 12:23:23.944685 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:23.944635 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14742h10m33.89261083s" Jul 15 12:23:25.104032 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:25.103941 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:25.104620 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.104085 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:25.104620 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:25.104499 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:25.104620 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.104607 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:25.104796 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:25.104663 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:25.104796 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.104734 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:25.525589 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:25.524982 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:25.525589 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.525178 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:25.525589 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.525241 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:29.52522135 +0000 UTC m=+10.075980863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:25.626187 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:25.626154 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:25.626360 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:25.626233 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:25.626431 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.626380 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:25.626431 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.626401 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:25.626431 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.626414 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:25.626591 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.626490 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:29.626448947 +0000 UTC m=+10.177208460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:25.626946 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.626910 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:25.627041 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:25.626963 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:23:29.626948631 +0000 UTC m=+10.177708149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:27.104398 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:27.103605 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:27.104398 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:27.103623 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:27.104398 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:27.103654 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:27.104398 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:27.103790 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:27.104398 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:27.103910 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:27.104398 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:27.104359 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:29.103954 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:29.103871 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:29.104375 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:29.103876 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:29.104375 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.103994 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:29.104375 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.104078 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:29.104375 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:29.103876 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:29.104375 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.104182 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:29.562367 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:29.562303 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:29.562536 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.562467 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:29.562610 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.562538 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:37.562517403 +0000 UTC m=+18.113276925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:29.662597 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:29.662677 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.662859 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.662884 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.662897 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.662954 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:37.662936098 +0000 UTC m=+18.213695610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.663353 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:29.663446 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:29.663404 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:23:37.663388505 +0000 UTC m=+18.214148017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:31.104662 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:31.104171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:31.104662 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:31.104183 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:31.104662 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:31.104183 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:31.104662 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:31.104339 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:31.104662 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:31.104475 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:31.104662 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:31.104555 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:33.103475 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:33.103423 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:33.103475 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:33.103458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:33.103475 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:33.103469 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:33.104027 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:33.103550 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:33.104027 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:33.103643 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:33.104027 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:33.103742 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:35.104003 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:35.103766 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:35.104399 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:35.103831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:35.104399 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:35.104089 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:35.104399 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:35.104163 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:35.104399 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:35.103847 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:35.104399 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:35.104268 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:37.104452 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:37.104411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:37.104957 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:37.104459 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:37.104957 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.104565 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:37.104957 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.104621 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:37.104957 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:37.104647 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:37.104957 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.104760 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:37.620712 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:37.620680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:37.620885 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.620791 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:37.620885 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.620862 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:53.620845771 +0000 UTC m=+34.171605296 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:37.721483 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:37.721438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:37.721633 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:37.721519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:37.721633 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.721591 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:37.721633 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.721611 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:37.721633 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.721622 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:37.721633 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.721631 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:37.721834 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.721679 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:53.721660227 +0000 UTC m=+34.272419737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:37.721834 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:37.721701 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:23:53.721688859 +0000 UTC m=+34.272448371 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:39.103985 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:39.103943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:39.103985 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:39.103980 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:39.104504 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:39.104046 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:39.104504 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:39.104064 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:39.104504 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:39.104185 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:39.104504 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:39.104285 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:41.104151 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:41.104117 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:41.104534 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:41.104231 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:41.104534 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:41.104238 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:41.104534 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:41.104259 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:41.104534 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:41.104330 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:41.104534 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:41.104404 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:42.221158 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.220127 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" event={"ID":"19c3823af9989bc31c938283a398f768","Type":"ContainerStarted","Data":"010f6cf3be24e1f8643a130feea47f0d226204f75740553175681268413853a2"} Jul 15 12:23:42.223759 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.223725 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"9c8c7a2f579787b8bd7d455ec650115910ee1a875d8630c782a79e4af43b0c7b"} Jul 15 12:23:42.225894 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.225830 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" event={"ID":"c9a8cf64-9dce-4bcf-aba1-b72ddde9cfaf","Type":"ContainerStarted","Data":"2c5f1e746ffce76a1e61f2925246d996e3d593e8044d538b5a8af0778fb80c40"} Jul 15 12:23:42.227462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.227435 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9f2dk" event={"ID":"c79cb6fd-6a97-4b80-8f34-5106f3096787","Type":"ContainerStarted","Data":"21f9dcca276ad16502d6527b95c92ae12381ffde5b04a85fa6cec359244b69dd"} Jul 15 12:23:42.246392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.246346 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-131.ec2.internal" podStartSLOduration=21.24632845 podStartE2EDuration="21.24632845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:42.245530671 +0000 UTC m=+22.796290204" watchObservedRunningTime="2026-07-15 12:23:42.24632845 +0000 UTC m=+22.797087983" Jul 15 12:23:42.271251 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.271197 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9f2dk" podStartSLOduration=2.60467868 podStartE2EDuration="22.271175743s" totalImagesPullingTime="19.666497063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:42.270027814 +0000 UTC m=+22.820787347" watchObservedRunningTime="2026-07-15 12:23:42.271175743 +0000 UTC m=+22.821935278" Jul 15 12:23:42.295912 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:42.295818 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lz2rl" podStartSLOduration=2.759787547 podStartE2EDuration="22.295787459s" totalImagesPullingTime="19.535999912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:42.295375305 +0000 UTC m=+22.846134865" watchObservedRunningTime="2026-07-15 12:23:42.295787459 +0000 UTC m=+22.846546990" Jul 15 12:23:43.104300 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.104114 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:43.104408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.104171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:43.104408 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:43.104394 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:43.104522 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.104171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:43.104522 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:43.104480 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:43.104611 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:43.104521 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:43.232542 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.232515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"7da6a0a00720b9e525dcf70119f85bdb4ed7ba8d38c51b93036011d42962e519"} Jul 15 12:23:43.233060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.232576 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"ae2794d229ec84488e1c275060c294cd499a206a581601670c2440a293adc2b1"} Jul 15 12:23:43.233060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.232621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"878a0f3691d9d1bfd1ab5683a4fa068280c25cb400adb8bd4614018aaf709cd6"} Jul 15 12:23:43.233060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.232652 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"3b4bd627fd4a0a99c7fc77eebdff186dd4e203fb298660ca3ee3e7c50713612e"} Jul 15 12:23:43.233060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.232703 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"f17e3d61e344ff57c82fec7d20661fec6e7744caa225ce3f5f1d3ec8db5ce513"} Jul 15 12:23:43.233773 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.233743 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xksn9" event={"ID":"90572c3f-6cb5-4c41-8b12-222f3041b25d","Type":"ContainerStarted","Data":"90de6bf96f69c4e20b7270dda83e1caabb5e10f000b64f6de897f13a1a9538e7"} Jul 15 12:23:43.235218 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.235196 2559 generic.go:356] "Generic (PLEG): container finished" podID="7848a9fc-b82c-4f02-9bcd-7e43fa24fbae" containerID="437078bb951fa9bea1c6979e191d55ed1eb10a5855fcc63d9bd1188c28923d2a" exitCode=0 Jul 15 12:23:43.235313 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.235256 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerDied","Data":"437078bb951fa9bea1c6979e191d55ed1eb10a5855fcc63d9bd1188c28923d2a"} Jul 15 12:23:43.236748 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.236710 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nr9tq" event={"ID":"489c2f9f-1652-4042-9366-dbb76fac0840","Type":"ContainerStarted","Data":"c5f722037261b5a61da4b1abe4255ee6a8af355c2b91679fc491dfe0062bdf97"} Jul 15 12:23:43.238249 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.238144 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m2qqp" event={"ID":"cdc00c95-fb22-4c7d-858d-f6aac54291cc","Type":"ContainerStarted","Data":"a164f6ab3d564d1ccdf130727b20ef9a757aa7d7582b1ecb5e7ce30f35396552"} Jul 15 12:23:43.239528 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.239508 2559 generic.go:356] "Generic (PLEG): container finished" podID="909893141fa7e6fa573a50328ab8fe38" containerID="baa51ecd5226fffab882faa702ecf5703d9d71a258f7e87718b43908501111d9" exitCode=0 Jul 15 12:23:43.239622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.239565 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" event={"ID":"909893141fa7e6fa573a50328ab8fe38","Type":"ContainerDied","Data":"baa51ecd5226fffab882faa702ecf5703d9d71a258f7e87718b43908501111d9"} Jul 15 12:23:43.240949 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.240916 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5bf7h" event={"ID":"67367022-d0b3-4781-a0f2-5e85dbb73156","Type":"ContainerStarted","Data":"43933280cfe8320035a460d0960e69c795164b2ab95cf63b524fa91e04549fd1"} Jul 15 12:23:43.242472 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.242452 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" event={"ID":"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff","Type":"ContainerStarted","Data":"9ca75efdb9a1659871ad5899c5e0b360a6a54e63bff11e22ffbfeb3d8052ade1"} Jul 15 12:23:43.257523 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.257484 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xksn9" podStartSLOduration=6.748217166 podStartE2EDuration="22.257472613s" totalImagesPullingTime="15.509255447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:43.257168744 +0000 UTC m=+23.807928276" watchObservedRunningTime="2026-07-15 12:23:43.257472613 +0000 UTC m=+23.808232145" Jul 15 12:23:43.311944 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.311908 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m2qqp" podStartSLOduration=11.921307689 podStartE2EDuration="23.311896744s" totalImagesPullingTime="11.390589055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:43.277788209 +0000 UTC m=+23.828547762" watchObservedRunningTime="2026-07-15 12:23:43.311896744 +0000 UTC m=+23.862656275" Jul 15 12:23:43.334924 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.334864 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5bf7h" podStartSLOduration=2.729002663 podStartE2EDuration="22.334853913s" totalImagesPullingTime="19.60585125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:43.334579952 +0000 UTC m=+23.885339483" watchObservedRunningTime="2026-07-15 12:23:43.334853913 +0000 UTC m=+23.885613445" Jul 15 12:23:43.415018 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.414967 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nr9tq" podStartSLOduration=12.040338451 podStartE2EDuration="23.414950263s" totalImagesPullingTime="11.374611812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:43.414939271 +0000 UTC m=+23.965698821" watchObservedRunningTime="2026-07-15 12:23:43.414950263 +0000 UTC m=+23.965709796" Jul 15 12:23:43.624011 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.623985 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 12:23:43.974132 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.974036 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T12:23:43.624004166Z","UUID":"9acaa7c5-feb1-4d03-8ef8-e92f81fa0566","Handler":null,"Name":"","Endpoint":""} Jul 15 12:23:43.975537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.975512 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 12:23:43.975651 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:43.975556 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 12:23:44.096294 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:44.096263 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:44.097052 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:44.097027 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:44.247177 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:44.247094 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" event={"ID":"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff","Type":"ContainerStarted","Data":"fd8de18608ae742ad0cc068438b72cc3f22cc3b8ad49f34c437f1ff352ad658d"} Jul 15 12:23:44.249216 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:44.249122 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" event={"ID":"909893141fa7e6fa573a50328ab8fe38","Type":"ContainerStarted","Data":"ceefb67e47562d05de8cfeefcf8afe40d114f9a62ec3f8785c5bcc2a108d1910"} Jul 15 12:23:45.104446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.104362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:45.104446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.104379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:45.104446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.104379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:45.104764 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:45.104488 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:45.104764 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:45.104597 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:45.104764 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:45.104682 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:45.253484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.253441 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" event={"ID":"ba71e6e6-8a29-4ffd-a64e-28d9f7509cff","Type":"ContainerStarted","Data":"48514b0ca49a672049c254603306d6bf164b9c6330267721c542a666df74ad37"} Jul 15 12:23:45.257203 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.257179 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 12:23:45.257355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.257209 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"8767f0746696dcbdc1a2aec1d4f709d2eb972bc231ef469a87d9b949aa650bab"} Jul 15 12:23:45.277631 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:45.277575 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-131.ec2.internal" podStartSLOduration=24.277560451 podStartE2EDuration="24.277560451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:44.26993194 +0000 UTC m=+24.820691473" watchObservedRunningTime="2026-07-15 12:23:45.277560451 +0000 UTC m=+25.828319983" Jul 15 12:23:46.623167 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:46.623128 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:46.623627 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:46.623250 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 12:23:46.623787 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:46.623729 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m2qqp" Jul 15 12:23:46.643582 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:46.643537 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7prtn" podStartSLOduration=3.783020235 podStartE2EDuration="25.643520164s" totalImagesPullingTime="21.860499929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:45.278904445 +0000 UTC m=+25.829663998" watchObservedRunningTime="2026-07-15 12:23:46.643520164 +0000 UTC m=+27.194279697" Jul 15 12:23:47.103642 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:47.103611 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:47.103843 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:47.103624 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:47.103843 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:47.103730 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:47.103843 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:47.103753 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:47.103993 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:47.103845 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:47.103993 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:47.103923 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:47.267103 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:47.266712 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" event={"ID":"b314c57b-6e4c-439a-80c3-caba783d364d","Type":"ContainerStarted","Data":"5bc365b95bf714fb5a165c4127874f420d2833d282c62e191e02b88b92010685"} Jul 15 12:23:47.318129 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:47.317991 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" podStartSLOduration=6.5123633210000005 podStartE2EDuration="26.317966355s" totalImagesPullingTime="19.805603034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:47.315822992 +0000 UTC m=+27.866582545" watchObservedRunningTime="2026-07-15 12:23:47.317966355 +0000 UTC m=+27.868725888" Jul 15 12:23:48.270139 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.270109 2559 generic.go:356] "Generic (PLEG): container finished" podID="7848a9fc-b82c-4f02-9bcd-7e43fa24fbae" containerID="beca40a115c8a201ab08227d285ac57990c88a9dd13ecd6d36d1ff85d6299477" exitCode=0 Jul 15 12:23:48.270948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.270172 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerDied","Data":"beca40a115c8a201ab08227d285ac57990c88a9dd13ecd6d36d1ff85d6299477"} Jul 15 12:23:48.270948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.270604 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:48.270948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.270672 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:48.270948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.270720 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:48.286232 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.286212 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:48.286351 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:48.286313 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:23:49.104081 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.103792 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:49.104288 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.103851 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:49.104288 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:49.104191 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:49.104288 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.103853 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:49.104288 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:49.104238 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:49.104434 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:49.104302 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:49.408755 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.407499 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qqkj"] Jul 15 12:23:49.408755 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.407672 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:49.408755 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:49.407796 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:49.413730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.413694 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dcp89"] Jul 15 12:23:49.413875 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.413814 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:49.413931 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:49.413893 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:49.417532 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.417505 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mbn5n"] Jul 15 12:23:49.417653 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:49.417606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:49.417698 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:49.417683 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:50.276194 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:50.276162 2559 generic.go:356] "Generic (PLEG): container finished" podID="7848a9fc-b82c-4f02-9bcd-7e43fa24fbae" containerID="1728f99593ace145c9f0e1b7e8331b737514ba37d240147e025aa59ee347445a" exitCode=0 Jul 15 12:23:50.276355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:50.276244 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerDied","Data":"1728f99593ace145c9f0e1b7e8331b737514ba37d240147e025aa59ee347445a"} Jul 15 12:23:51.103843 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:51.103793 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:51.104204 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:51.103793 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:51.104204 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:51.103919 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:51.104204 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:51.103798 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:51.104204 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:51.104016 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:51.104204 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:51.104103 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:52.282722 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:52.282685 2559 generic.go:356] "Generic (PLEG): container finished" podID="7848a9fc-b82c-4f02-9bcd-7e43fa24fbae" containerID="bdf915da2b85626444df70582cf0be65c68b98d7b213f5b816f0fc172898ea0d" exitCode=0 Jul 15 12:23:52.283194 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:52.282731 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerDied","Data":"bdf915da2b85626444df70582cf0be65c68b98d7b213f5b816f0fc172898ea0d"} Jul 15 12:23:53.104108 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:53.104072 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:53.104108 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:53.104089 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:53.104323 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:53.104090 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:53.104323 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.104215 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcp89" podUID="09106fe3-c171-4e91-b958-ba44ff83852e" Jul 15 12:23:53.104323 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.104292 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbn5n" podUID="080bb3f7-f2f2-4417-845b-519ee121d45e" Jul 15 12:23:53.104472 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.104374 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qqkj" podUID="741e3629-2b48-4694-a392-189d31991a8f" Jul 15 12:23:53.635278 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:53.635244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:53.635670 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.635363 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:53.635670 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.635423 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs podName:09106fe3-c171-4e91-b958-ba44ff83852e nodeName:}" failed. No retries permitted until 2026-07-15 12:24:25.635409524 +0000 UTC m=+66.186169037 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs") pod "network-metrics-daemon-dcp89" (UID: "09106fe3-c171-4e91-b958-ba44ff83852e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:23:53.736605 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:53.736568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:53.736773 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:53.736627 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:53.736773 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.736734 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:53.736773 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.736751 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:23:53.736773 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.736769 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:23:53.736994 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.736783 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bzl9b for pod openshift-network-diagnostics/network-check-target-mbn5n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:53.736994 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.736827 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret podName:741e3629-2b48-4694-a392-189d31991a8f nodeName:}" failed. No retries permitted until 2026-07-15 12:24:25.736789361 +0000 UTC m=+66.287548873 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret") pod "global-pull-secret-syncer-5qqkj" (UID: "741e3629-2b48-4694-a392-189d31991a8f") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:23:53.736994 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:53.736855 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b podName:080bb3f7-f2f2-4417-845b-519ee121d45e nodeName:}" failed. No retries permitted until 2026-07-15 12:24:25.736838454 +0000 UTC m=+66.287597978 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzl9b" (UniqueName: "kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b") pod "network-check-target-mbn5n" (UID: "080bb3f7-f2f2-4417-845b-519ee121d45e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:23:54.778701 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.778638 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-131.ec2.internal" event="NodeReady" Jul 15 12:23:54.779055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.778766 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 12:23:54.834977 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.834947 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64f9cf7dd9-nqzbp"] Jul 15 12:23:54.858087 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.857825 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.859080 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.858993 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f9cf7dd9-nqzbp"] Jul 15 12:23:54.861789 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.861766 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 12:23:54.862324 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.862306 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4zvck\"" Jul 15 12:23:54.863295 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.863113 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 12:23:54.864431 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.864407 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 12:23:54.867858 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.867123 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jm9h6"] Jul 15 12:23:54.868658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.868640 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 12:23:54.881632 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.881611 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8tw9h"] Jul 15 12:23:54.881791 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.881764 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:54.885635 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.885605 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 12:23:54.885635 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.885625 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ggrzt\"" Jul 15 12:23:54.885797 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.885667 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 12:23:54.895721 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.895704 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jm9h6"] Jul 15 12:23:54.895847 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.895743 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8tw9h"] Jul 15 12:23:54.895903 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.895850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:54.899619 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.899602 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 12:23:54.900006 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.899989 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gd95d\"" Jul 15 12:23:54.900096 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.899989 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 12:23:54.900275 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.900261 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 12:23:54.945935 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.945910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-bound-sa-token\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.945966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-ca-trust-extracted\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.946023 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.946049 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-installation-pull-secrets\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.946086 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsh2z\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-kube-api-access-gsh2z\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.946144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-certificates\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.946180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-trusted-ca\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:54.946354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:54.946252 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-image-registry-private-configuration\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.046871 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.046766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4hl\" (UniqueName: \"kubernetes.io/projected/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-kube-api-access-zc4hl\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.046871 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.046829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.046882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-installation-pull-secrets\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.046908 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsh2z\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-kube-api-access-gsh2z\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.046934 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f35b6a0c-30d5-486c-a98b-adb08a528ea7-cert\") pod \"ingress-canary-8tw9h\" (UID: \"f35b6a0c-30d5-486c-a98b-adb08a528ea7\") " pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.046956 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.046982 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f9cf7dd9-nqzbp: secret "image-registry-tls" not found Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.046992 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-certificates\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047019 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-trusted-ca\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047070 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.047040 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls podName:e076e1be-3bea-4ee7-a70c-ee8f10a4bf98 nodeName:}" failed. No retries permitted until 2026-07-15 12:23:55.547023173 +0000 UTC m=+36.097782702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls") pod "image-registry-64f9cf7dd9-nqzbp" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98") : secret "image-registry-tls" not found Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047114 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-image-registry-private-configuration\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047141 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-bound-sa-token\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047168 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-config-volume\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-ca-trust-extracted\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047255 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2rd\" (UniqueName: \"kubernetes.io/projected/f35b6a0c-30d5-486c-a98b-adb08a528ea7-kube-api-access-9j2rd\") pod \"ingress-canary-8tw9h\" (UID: \"f35b6a0c-30d5-486c-a98b-adb08a528ea7\") " pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.047408 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.047290 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-tmp-dir\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.048103 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.048040 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-ca-trust-extracted\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.048103 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.048069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-certificates\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.048274 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.048181 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-trusted-ca\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.052127 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.052010 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-installation-pull-secrets\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.052220 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.052032 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-image-registry-private-configuration\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.060182 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.060157 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-bound-sa-token\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.060329 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.060307 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsh2z\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-kube-api-access-gsh2z\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.104014 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.103978 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:23:55.104181 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.104030 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:23:55.104181 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.104035 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:23:55.109785 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.109757 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x2twz\"" Jul 15 12:23:55.109942 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.109826 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 12:23:55.109942 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.109855 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 12:23:55.110094 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.110074 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 12:23:55.110094 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.110089 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 12:23:55.110266 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.110102 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qvxmk\"" Jul 15 12:23:55.116909 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.116869 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:23:55.147700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.147677 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2rd\" (UniqueName: \"kubernetes.io/projected/f35b6a0c-30d5-486c-a98b-adb08a528ea7-kube-api-access-9j2rd\") pod \"ingress-canary-8tw9h\" (UID: \"f35b6a0c-30d5-486c-a98b-adb08a528ea7\") " pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.147838 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.147725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-tmp-dir\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.147838 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.147764 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4hl\" (UniqueName: \"kubernetes.io/projected/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-kube-api-access-zc4hl\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.147995 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.147958 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f35b6a0c-30d5-486c-a98b-adb08a528ea7-cert\") pod \"ingress-canary-8tw9h\" (UID: \"f35b6a0c-30d5-486c-a98b-adb08a528ea7\") " pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.148058 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.148041 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.148109 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.148074 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-config-volume\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.148149 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.148128 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-tmp-dir\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.148203 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.148165 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 12:23:55.148252 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.148232 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls podName:62c2d3bc-8cac-427f-a8da-bec575ab8b3e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:55.648209031 +0000 UTC m=+36.198968555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls") pod "dns-default-jm9h6" (UID: "62c2d3bc-8cac-427f-a8da-bec575ab8b3e") : secret "dns-default-metrics-tls" not found Jul 15 12:23:55.148504 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.148487 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-config-volume\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.150566 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.150548 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f35b6a0c-30d5-486c-a98b-adb08a528ea7-cert\") pod \"ingress-canary-8tw9h\" (UID: \"f35b6a0c-30d5-486c-a98b-adb08a528ea7\") " pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.162276 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.162255 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4hl\" (UniqueName: \"kubernetes.io/projected/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-kube-api-access-zc4hl\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.163116 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.163097 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2rd\" (UniqueName: \"kubernetes.io/projected/f35b6a0c-30d5-486c-a98b-adb08a528ea7-kube-api-access-9j2rd\") pod \"ingress-canary-8tw9h\" (UID: \"f35b6a0c-30d5-486c-a98b-adb08a528ea7\") " pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.205316 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.205282 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8tw9h" Jul 15 12:23:55.232678 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.232647 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:23:55.550603 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.550568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:55.550792 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.550740 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 12:23:55.550792 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.550763 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f9cf7dd9-nqzbp: secret "image-registry-tls" not found Jul 15 12:23:55.550900 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.550837 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls podName:e076e1be-3bea-4ee7-a70c-ee8f10a4bf98 nodeName:}" failed. No retries permitted until 2026-07-15 12:23:56.550820248 +0000 UTC m=+37.101579762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls") pod "image-registry-64f9cf7dd9-nqzbp" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98") : secret "image-registry-tls" not found Jul 15 12:23:55.651231 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:55.651194 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:55.651399 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.651328 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 12:23:55.651399 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:55.651393 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls podName:62c2d3bc-8cac-427f-a8da-bec575ab8b3e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:56.651375687 +0000 UTC m=+37.202135200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls") pod "dns-default-jm9h6" (UID: "62c2d3bc-8cac-427f-a8da-bec575ab8b3e") : secret "dns-default-metrics-tls" not found Jul 15 12:23:56.474592 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:56.474570 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8tw9h"] Jul 15 12:23:56.538839 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:23:56.538791 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35b6a0c_30d5_486c_a98b_adb08a528ea7.slice/crio-1dee98c351be80e5b9015e0fe81ab67608e36ceb959786ba51fabc243d2496cb WatchSource:0}: Error finding container 1dee98c351be80e5b9015e0fe81ab67608e36ceb959786ba51fabc243d2496cb: Status 404 returned error can't find the container with id 1dee98c351be80e5b9015e0fe81ab67608e36ceb959786ba51fabc243d2496cb Jul 15 12:23:56.556686 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:56.556662 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:56.556812 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:56.556790 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 12:23:56.556855 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:56.556815 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f9cf7dd9-nqzbp: secret "image-registry-tls" not found Jul 15 12:23:56.556885 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:56.556856 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls podName:e076e1be-3bea-4ee7-a70c-ee8f10a4bf98 nodeName:}" failed. No retries permitted until 2026-07-15 12:23:58.556843638 +0000 UTC m=+39.107603148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls") pod "image-registry-64f9cf7dd9-nqzbp" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98") : secret "image-registry-tls" not found Jul 15 12:23:56.657253 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:56.657227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:56.657358 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:56.657344 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 12:23:56.657409 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:56.657390 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls podName:62c2d3bc-8cac-427f-a8da-bec575ab8b3e nodeName:}" failed. No retries permitted until 2026-07-15 12:23:58.657377537 +0000 UTC m=+39.208137046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls") pod "dns-default-jm9h6" (UID: "62c2d3bc-8cac-427f-a8da-bec575ab8b3e") : secret "dns-default-metrics-tls" not found Jul 15 12:23:57.300257 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:57.300213 2559 generic.go:356] "Generic (PLEG): container finished" podID="7848a9fc-b82c-4f02-9bcd-7e43fa24fbae" containerID="204e01d7428a5665121433486d9515924ad8067b3b596a553e106f22057f42d6" exitCode=0 Jul 15 12:23:57.300434 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:57.300310 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerDied","Data":"204e01d7428a5665121433486d9515924ad8067b3b596a553e106f22057f42d6"} Jul 15 12:23:57.301458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:57.301435 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8tw9h" event={"ID":"f35b6a0c-30d5-486c-a98b-adb08a528ea7","Type":"ContainerStarted","Data":"1dee98c351be80e5b9015e0fe81ab67608e36ceb959786ba51fabc243d2496cb"} Jul 15 12:23:58.306372 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:58.306144 2559 generic.go:356] "Generic (PLEG): container finished" podID="7848a9fc-b82c-4f02-9bcd-7e43fa24fbae" containerID="e12f3e0872e58c9e7616a96dff1c43e76575e6d05c4d4d93879c38855b5843f9" exitCode=0 Jul 15 12:23:58.306785 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:58.306214 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerDied","Data":"e12f3e0872e58c9e7616a96dff1c43e76575e6d05c4d4d93879c38855b5843f9"} Jul 15 12:23:58.572131 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:58.572018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:23:58.572269 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:58.572166 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 12:23:58.572269 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:58.572180 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f9cf7dd9-nqzbp: secret "image-registry-tls" not found Jul 15 12:23:58.572269 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:58.572231 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls podName:e076e1be-3bea-4ee7-a70c-ee8f10a4bf98 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.572217822 +0000 UTC m=+43.122977348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls") pod "image-registry-64f9cf7dd9-nqzbp" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98") : secret "image-registry-tls" not found Jul 15 12:23:58.672981 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:58.672942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:23:58.673140 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:58.673123 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 12:23:58.673218 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:23:58.673206 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls podName:62c2d3bc-8cac-427f-a8da-bec575ab8b3e nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.67318343 +0000 UTC m=+43.223942964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls") pod "dns-default-jm9h6" (UID: "62c2d3bc-8cac-427f-a8da-bec575ab8b3e") : secret "dns-default-metrics-tls" not found Jul 15 12:23:59.312309 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:59.312272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfkst" event={"ID":"7848a9fc-b82c-4f02-9bcd-7e43fa24fbae","Type":"ContainerStarted","Data":"8f7fcb9c8924293c1ff6ae495815f8fcc0abdc22c119d2f021a8851655e0555e"} Jul 15 12:23:59.344797 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:59.344738 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qfkst" podStartSLOduration=9.233203094 podStartE2EDuration="39.344721581s" totalImagesPullingTime="30.111518487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:59.343215706 +0000 UTC m=+39.893975259" watchObservedRunningTime="2026-07-15 12:23:59.344721581 +0000 UTC m=+39.895481112" Jul 15 12:23:59.486270 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:59.486235 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:23:59.582794 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:59.582760 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:23:59.782912 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:59.782875 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:23:59.942616 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:23:59.942526 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:24:00.315962 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.315919 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8tw9h" event={"ID":"f35b6a0c-30d5-486c-a98b-adb08a528ea7","Type":"ContainerStarted","Data":"2f349768ef73cd31e46b49f9006181114948c91cbab39d33497d9992ff425cca"} Jul 15 12:24:00.341835 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.341770 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8tw9h" podStartSLOduration=3.3174555039999998 podStartE2EDuration="6.341756001s" totalImagesPullingTime="3.024300497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:00.341277661 +0000 UTC m=+40.892037174" watchObservedRunningTime="2026-07-15 12:24:00.341756001 +0000 UTC m=+40.892515532" Jul 15 12:24:00.360938 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.360908 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" limit=1024 Jul 15 12:24:00.708296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.708218 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8"] Jul 15 12:24:00.711337 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.711317 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.714444 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.714419 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c"] Jul 15 12:24:00.715851 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.715820 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 12:24:00.716076 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.716063 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qfbhz\"" Jul 15 12:24:00.716970 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.716954 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" Jul 15 12:24:00.717165 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.717142 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 12:24:00.717305 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.717204 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 12:24:00.718151 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.718131 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:00.721526 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.721511 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 12:24:00.722062 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.722047 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-fwzjb\"" Jul 15 12:24:00.734590 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.734575 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:00.749956 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.749936 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9"] Jul 15 12:24:00.752755 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.752739 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8"] Jul 15 12:24:00.752846 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.752764 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79c7ddc6b-857qm"] Jul 15 12:24:00.752884 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.752871 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.755582 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.755563 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c"] Jul 15 12:24:00.755656 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.755636 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.760732 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.760714 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 12:24:00.760908 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.760894 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 12:24:00.761845 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.761831 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 12:24:00.761928 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.761892 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 12:24:00.761980 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.761943 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 12:24:00.762055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.762040 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zkfkj\"" Jul 15 12:24:00.763526 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.763508 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 12:24:00.764169 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.764139 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 12:24:00.764259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.764181 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 12:24:00.764259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.764182 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 12:24:00.764259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.764212 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6r7rb\"" Jul 15 12:24:00.764259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.764217 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 12:24:00.778886 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.778862 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9"] Jul 15 12:24:00.779505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.779484 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79c7ddc6b-857qm"] Jul 15 12:24:00.889385 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.889356 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-xg6rc"] Jul 15 12:24:00.891515 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891495 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xrhz\" (UniqueName: \"kubernetes.io/projected/97c63a62-8a37-4495-90c9-0944d7fdb759-kube-api-access-8xrhz\") pod \"volume-data-source-validator-78495597f6-p5s6c\" (UID: \"97c63a62-8a37-4495-90c9-0944d7fdb759\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" Jul 15 12:24:00.891584 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891524 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-stats-auth\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.891584 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-default-certificate\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.891701 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891584 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a523205d-1b6d-4987-a42a-d3de5c5481de-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.891701 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891659 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.891825 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891702 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbkdg\" (UniqueName: \"kubernetes.io/projected/a523205d-1b6d-4987-a42a-d3de5c5481de-kube-api-access-hbkdg\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.891825 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.891825 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891765 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8jg5\" (UniqueName: \"kubernetes.io/projected/1effb694-29c1-4b54-8822-604b929a4284-kube-api-access-b8jg5\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.891825 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891794 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bbdl\" (UniqueName: \"kubernetes.io/projected/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-kube-api-access-4bbdl\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.892004 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891848 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.892004 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891872 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-serving-cert\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.892004 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.891894 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-config\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.892176 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.892158 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b"] Jul 15 12:24:00.892317 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.892300 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:00.895149 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.895126 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5"] Jul 15 12:24:00.895248 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.895226 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" Jul 15 12:24:00.898168 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.898151 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:00.903006 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.902984 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"serving-cert\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" type="*v1.Secret" Jul 15 12:24:00.914192 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914161 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" type="*v1.ConfigMap" Jul 15 12:24:00.914192 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914162 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 15 12:24:00.914342 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914162 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"console-operator-config\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" type="*v1.ConfigMap" Jul 15 12:24:00.914417 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914390 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 15 12:24:00.914417 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914400 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"samples-operator-tls\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" Jul 15 12:24:00.914520 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914416 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 15 12:24:00.914520 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914446 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-samples-operator-dockercfg-zs8dd\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zs8dd\"" type="*v1.Secret" Jul 15 12:24:00.914520 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914469 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 15 12:24:00.914677 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.914541 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-operator-dockercfg-bcqn9\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console-operator\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bcqn9\"" type="*v1.Secret" Jul 15 12:24:00.915276 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.915260 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg"] Jul 15 12:24:00.915943 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.915924 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"network-diagnostics-dockercfg-bwlhd\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-diagnostics\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bwlhd\"" type="*v1.Secret" Jul 15 12:24:00.918041 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.918027 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:00.921886 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.921867 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 12:24:00.922082 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.922067 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 12:24:00.922165 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.922147 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6b57l\"" Jul 15 12:24:00.922289 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.922268 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 12:24:00.922392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.922292 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:00.923029 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.923010 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b"] Jul 15 12:24:00.939048 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.939028 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-xg6rc"] Jul 15 12:24:00.965075 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.965056 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5"] Jul 15 12:24:00.965894 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.965876 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg"] Jul 15 12:24:00.979677 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.979657 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-bmp27"] Jul 15 12:24:00.982471 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.982449 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:00.992634 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bbdl\" (UniqueName: \"kubernetes.io/projected/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-kube-api-access-4bbdl\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.992728 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992648 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 12:24:00.992728 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992658 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 12:24:00.992728 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992664 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.992728 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992704 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7be01789-354a-4605-ab90-c7813324c30d-config\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:00.992890 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xrhz\" (UniqueName: \"kubernetes.io/projected/97c63a62-8a37-4495-90c9-0944d7fdb759-kube-api-access-8xrhz\") pod \"volume-data-source-validator-78495597f6-p5s6c\" (UID: \"97c63a62-8a37-4495-90c9-0944d7fdb759\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" Jul 15 12:24:00.992890 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992857 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-serving-cert\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.992948 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.992884 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:01.492867155 +0000 UTC m=+42.043626680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : configmap references non-existent config key: service-ca.crt Jul 15 12:24:00.992948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-config\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.993039 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992953 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.993039 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992964 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rtdkw\"" Jul 15 12:24:00.993039 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.992978 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbkdg\" (UniqueName: \"kubernetes.io/projected/a523205d-1b6d-4987-a42a-d3de5c5481de-kube-api-access-hbkdg\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.993039 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.993059 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-stats-auth\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.993129 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls podName:a523205d-1b6d-4987-a42a-d3de5c5481de nodeName:}" failed. No retries permitted until 2026-07-15 12:24:01.493109438 +0000 UTC m=+42.043868956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wpxm9" (UID: "a523205d-1b6d-4987-a42a-d3de5c5481de") : secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.993166 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-default-certificate\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsgks\" (UniqueName: \"kubernetes.io/projected/7be01789-354a-4605-ab90-c7813324c30d-kube-api-access-wsgks\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:00.993222 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:00.993213 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:01.493197959 +0000 UTC m=+42.043957491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : secret "router-metrics-certs-default" not found Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993305 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a523205d-1b6d-4987-a42a-d3de5c5481de-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993341 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9gt4\" (UniqueName: \"kubernetes.io/projected/d3e9dae6-5079-4972-b9b2-c5d72c7351ad-kube-api-access-s9gt4\") pod \"network-check-source-869dcfcdcd-nst9b\" (UID: \"d3e9dae6-5079-4972-b9b2-c5d72c7351ad\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrt8\" (UniqueName: \"kubernetes.io/projected/39e72a12-f0d8-4463-9c9e-5c48da5717c7-kube-api-access-nvrt8\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993421 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7be01789-354a-4605-ab90-c7813324c30d-serving-cert\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993444 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7be01789-354a-4605-ab90-c7813324c30d-trusted-ca\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993488 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8jg5\" (UniqueName: \"kubernetes.io/projected/1effb694-29c1-4b54-8822-604b929a4284-kube-api-access-b8jg5\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.993518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.993493 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-config\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.994184 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.994000 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 12:24:00.994255 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.994220 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a523205d-1b6d-4987-a42a-d3de5c5481de-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:00.994255 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.994249 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 12:24:00.996500 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.996476 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-serving-cert\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:00.996788 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.996773 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-default-certificate\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:00.996883 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:00.996865 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-stats-auth\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:01.010613 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.010592 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 12:24:01.012592 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.012575 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-bmp27"] Jul 15 12:24:01.021940 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.021924 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bbdl\" (UniqueName: \"kubernetes.io/projected/7eadd2a4-e5a7-4ad9-9689-83437e981ffd-kube-api-access-4bbdl\") pod \"service-ca-operator-7695fc6c8c-qszx8\" (UID: \"7eadd2a4-e5a7-4ad9-9689-83437e981ffd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:01.029323 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.029304 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbkdg\" (UniqueName: \"kubernetes.io/projected/a523205d-1b6d-4987-a42a-d3de5c5481de-kube-api-access-hbkdg\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:01.031630 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.031609 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xrhz\" (UniqueName: \"kubernetes.io/projected/97c63a62-8a37-4495-90c9-0944d7fdb759-kube-api-access-8xrhz\") pod \"volume-data-source-validator-78495597f6-p5s6c\" (UID: \"97c63a62-8a37-4495-90c9-0944d7fdb759\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" Jul 15 12:24:01.033660 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.033637 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8jg5\" (UniqueName: \"kubernetes.io/projected/1effb694-29c1-4b54-8822-604b929a4284-kube-api-access-b8jg5\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:01.094836 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094816 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-snapshots\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.094920 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.094920 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094864 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-serving-cert\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.094920 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094915 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsgks\" (UniqueName: \"kubernetes.io/projected/7be01789-354a-4605-ab90-c7813324c30d-kube-api-access-wsgks\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:01.095019 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9gt4\" (UniqueName: \"kubernetes.io/projected/d3e9dae6-5079-4972-b9b2-c5d72c7351ad-kube-api-access-s9gt4\") pod \"network-check-source-869dcfcdcd-nst9b\" (UID: \"d3e9dae6-5079-4972-b9b2-c5d72c7351ad\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" Jul 15 12:24:01.095019 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrt8\" (UniqueName: \"kubernetes.io/projected/39e72a12-f0d8-4463-9c9e-5c48da5717c7-kube-api-access-nvrt8\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:01.095019 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.094993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:01.095112 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7be01789-354a-4605-ab90-c7813324c30d-serving-cert\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:01.095112 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7be01789-354a-4605-ab90-c7813324c30d-trusted-ca\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:01.095112 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-tmp\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.095211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095128 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7be01789-354a-4605-ab90-c7813324c30d-config\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:01.095211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095159 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-584qr\" (UniqueName: \"kubernetes.io/projected/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-kube-api-access-584qr\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.095211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095187 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh5st\" (UniqueName: \"kubernetes.io/projected/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-kube-api-access-xh5st\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.095310 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.095310 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095240 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.095380 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.095313 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.117820 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.117781 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9gt4\" (UniqueName: \"kubernetes.io/projected/d3e9dae6-5079-4972-b9b2-c5d72c7351ad-kube-api-access-s9gt4\") pod \"network-check-source-869dcfcdcd-nst9b\" (UID: \"d3e9dae6-5079-4972-b9b2-c5d72c7351ad\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" Jul 15 12:24:01.196067 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196048 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-584qr\" (UniqueName: \"kubernetes.io/projected/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-kube-api-access-584qr\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.196165 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196074 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5st\" (UniqueName: \"kubernetes.io/projected/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-kube-api-access-xh5st\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.196165 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196096 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.196165 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.196259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.196348 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-snapshots\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.196413 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196370 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.196413 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196388 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-serving-cert\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.196498 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-tmp\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.196670 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196651 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.196977 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.196954 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-tmp\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.197087 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.197068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-snapshots\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.197366 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.197345 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.197843 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.197828 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.198379 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.198364 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.198784 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.198763 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-serving-cert\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.217536 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.217483 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5st\" (UniqueName: \"kubernetes.io/projected/19bf4b2b-2ac1-4b62-a476-df4f617e74f8-kube-api-access-xh5st\") pod \"kube-storage-version-migrator-operator-799cb7f958-dp9wg\" (UID: \"19bf4b2b-2ac1-4b62-a476-df4f617e74f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.218430 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.218414 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-584qr\" (UniqueName: \"kubernetes.io/projected/1e360f0c-fc2b-4915-92ab-61a3c4b63e71-kube-api-access-584qr\") pod \"insights-operator-68d9dd94cf-bmp27\" (UID: \"1e360f0c-fc2b-4915-92ab-61a3c4b63e71\") " pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.225488 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.225474 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" Jul 15 12:24:01.290752 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.290720 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" Jul 15 12:24:01.320584 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.320425 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" Jul 15 12:24:01.326486 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.326463 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" Jul 15 12:24:01.360699 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.360633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg"] Jul 15 12:24:01.367322 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:01.366750 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19bf4b2b_2ac1_4b62_a476_df4f617e74f8.slice/crio-3a846edfd513184116cc490c4b42ec545fbe940f431fbfe933f68e202ab841b3 WatchSource:0}: Error finding container 3a846edfd513184116cc490c4b42ec545fbe940f431fbfe933f68e202ab841b3: Status 404 returned error can't find the container with id 3a846edfd513184116cc490c4b42ec545fbe940f431fbfe933f68e202ab841b3 Jul 15 12:24:01.470281 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.470215 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-bmp27"] Jul 15 12:24:01.473852 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:01.473821 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e360f0c_fc2b_4915_92ab_61a3c4b63e71.slice/crio-2421e271bffffc5ff0adc71791bd02487961de32b40edbf1cf3749942c1271d0 WatchSource:0}: Error finding container 2421e271bffffc5ff0adc71791bd02487961de32b40edbf1cf3749942c1271d0: Status 404 returned error can't find the container with id 2421e271bffffc5ff0adc71791bd02487961de32b40edbf1cf3749942c1271d0 Jul 15 12:24:01.490334 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.490311 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8"] Jul 15 12:24:01.492977 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:01.492952 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eadd2a4_e5a7_4ad9_9689_83437e981ffd.slice/crio-30326e915c40f168cd26c03b6ad4e8d634cc5f667ed7f54e12f6bdf06205aff7 WatchSource:0}: Error finding container 30326e915c40f168cd26c03b6ad4e8d634cc5f667ed7f54e12f6bdf06205aff7: Status 404 returned error can't find the container with id 30326e915c40f168cd26c03b6ad4e8d634cc5f667ed7f54e12f6bdf06205aff7 Jul 15 12:24:01.499560 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.499538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:01.499639 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.499584 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:01.499639 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.499602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:01.499732 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:01.499707 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 12:24:01.499732 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:01.499716 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.499694172 +0000 UTC m=+43.050453689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : configmap references non-existent config key: service-ca.crt Jul 15 12:24:01.499866 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:01.499739 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:01.499866 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:01.499760 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.499744873 +0000 UTC m=+43.050504384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : secret "router-metrics-certs-default" not found Jul 15 12:24:01.499866 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:01.499828 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls podName:a523205d-1b6d-4987-a42a-d3de5c5481de nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.499796324 +0000 UTC m=+43.050555839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wpxm9" (UID: "a523205d-1b6d-4987-a42a-d3de5c5481de") : secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:01.512970 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.512951 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c"] Jul 15 12:24:01.514591 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:01.514574 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c63a62_8a37_4495_90c9_0944d7fdb759.slice/crio-bf40edcdf8df9858f4979c6e36ee3ad4b877d72ae90b6276a9664928008eda6d WatchSource:0}: Error finding container bf40edcdf8df9858f4979c6e36ee3ad4b877d72ae90b6276a9664928008eda6d: Status 404 returned error can't find the container with id bf40edcdf8df9858f4979c6e36ee3ad4b877d72ae90b6276a9664928008eda6d Jul 15 12:24:01.751191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.751106 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 12:24:01.756727 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.756708 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7be01789-354a-4605-ab90-c7813324c30d-trusted-ca\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:01.768734 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.768711 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bwlhd\"" Jul 15 12:24:01.768837 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.768751 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:01.771912 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.771894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" Jul 15 12:24:01.773256 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.773243 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 12:24:01.822950 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.822924 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zs8dd\"" Jul 15 12:24:01.859246 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.859224 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 12:24:01.868507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.868483 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrt8\" (UniqueName: \"kubernetes.io/projected/39e72a12-f0d8-4463-9c9e-5c48da5717c7-kube-api-access-nvrt8\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:01.916720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.916696 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b"] Jul 15 12:24:01.924637 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.924618 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 12:24:01.926066 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:01.926044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7be01789-354a-4605-ab90-c7813324c30d-config\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:02.095667 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.095405 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: failed to sync secret cache: timed out waiting for the condition Jul 15 12:24:02.095667 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.095489 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls podName:39e72a12-f0d8-4463-9c9e-5c48da5717c7 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.595466552 +0000 UTC m=+43.146226081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls") pod "cluster-samples-operator-65db858894-g22p5" (UID: "39e72a12-f0d8-4463-9c9e-5c48da5717c7") : failed to sync secret cache: timed out waiting for the condition Jul 15 12:24:02.095667 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.095417 2559 secret.go:189] Couldn't get secret openshift-console-operator/serving-cert: failed to sync secret cache: timed out waiting for the condition Jul 15 12:24:02.095667 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.095568 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be01789-354a-4605-ab90-c7813324c30d-serving-cert podName:7be01789-354a-4605-ab90-c7813324c30d nodeName:}" failed. No retries permitted until 2026-07-15 12:24:02.595550364 +0000 UTC m=+43.146309880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/7be01789-354a-4605-ab90-c7813324c30d-serving-cert") pod "console-operator-7866497494-xg6rc" (UID: "7be01789-354a-4605-ab90-c7813324c30d") : failed to sync secret cache: timed out waiting for the condition Jul 15 12:24:02.294972 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.294754 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 12:24:02.296710 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.296534 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bcqn9\"" Jul 15 12:24:02.326876 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.326795 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" event={"ID":"19bf4b2b-2ac1-4b62-a476-df4f617e74f8","Type":"ContainerStarted","Data":"3a846edfd513184116cc490c4b42ec545fbe940f431fbfe933f68e202ab841b3"} Jul 15 12:24:02.328555 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.328519 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" event={"ID":"7eadd2a4-e5a7-4ad9-9689-83437e981ffd","Type":"ContainerStarted","Data":"30326e915c40f168cd26c03b6ad4e8d634cc5f667ed7f54e12f6bdf06205aff7"} Jul 15 12:24:02.330761 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.330709 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" event={"ID":"d3e9dae6-5079-4972-b9b2-c5d72c7351ad","Type":"ContainerStarted","Data":"3a563fbf1789f70c9fe10db12cfe48b74f161124d28d7b4f25c9006a1f49b43c"} Jul 15 12:24:02.332221 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.332147 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" event={"ID":"97c63a62-8a37-4495-90c9-0944d7fdb759","Type":"ContainerStarted","Data":"bf40edcdf8df9858f4979c6e36ee3ad4b877d72ae90b6276a9664928008eda6d"} Jul 15 12:24:02.335849 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.335770 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" event={"ID":"1e360f0c-fc2b-4915-92ab-61a3c4b63e71","Type":"ContainerStarted","Data":"2421e271bffffc5ff0adc71791bd02487961de32b40edbf1cf3749942c1271d0"} Jul 15 12:24:02.445425 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.445337 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:02.457993 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.457960 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsgks\" (UniqueName: \"kubernetes.io/projected/7be01789-354a-4605-ab90-c7813324c30d-kube-api-access-wsgks\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:02.507826 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.507766 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.511229 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.511289 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.511485 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.511501 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.511559 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls podName:a523205d-1b6d-4987-a42a-d3de5c5481de nodeName:}" failed. No retries permitted until 2026-07-15 12:24:04.511542261 +0000 UTC m=+45.062301786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wpxm9" (UID: "a523205d-1b6d-4987-a42a-d3de5c5481de") : secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.511677 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:04.511664544 +0000 UTC m=+45.062424060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : configmap references non-existent config key: service-ca.crt Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.511778 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 12:24:02.511890 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.511833 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:04.511820664 +0000 UTC m=+45.062580178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : secret "router-metrics-certs-default" not found Jul 15 12:24:02.612658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.612629 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:02.612658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.612662 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7be01789-354a-4605-ab90-c7813324c30d-serving-cert\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:02.612909 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.612693 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:02.612909 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.612793 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 12:24:02.612909 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.612836 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 12:24:02.612909 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.612847 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f9cf7dd9-nqzbp: secret "image-registry-tls" not found Jul 15 12:24:02.612909 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.612879 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls podName:39e72a12-f0d8-4463-9c9e-5c48da5717c7 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:03.612858413 +0000 UTC m=+44.163617922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls") pod "cluster-samples-operator-65db858894-g22p5" (UID: "39e72a12-f0d8-4463-9c9e-5c48da5717c7") : secret "samples-operator-tls" not found Jul 15 12:24:02.612909 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.612901 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls podName:e076e1be-3bea-4ee7-a70c-ee8f10a4bf98 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:10.612891087 +0000 UTC m=+51.163650600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls") pod "image-registry-64f9cf7dd9-nqzbp" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98") : secret "image-registry-tls" not found Jul 15 12:24:02.616594 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.615550 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7be01789-354a-4605-ab90-c7813324c30d-serving-cert\") pod \"console-operator-7866497494-xg6rc\" (UID: \"7be01789-354a-4605-ab90-c7813324c30d\") " pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:02.701887 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.701798 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:02.713682 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:02.713655 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:02.713862 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.713837 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 12:24:02.713926 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:02.713908 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls podName:62c2d3bc-8cac-427f-a8da-bec575ab8b3e nodeName:}" failed. No retries permitted until 2026-07-15 12:24:10.713885091 +0000 UTC m=+51.264644618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls") pod "dns-default-jm9h6" (UID: "62c2d3bc-8cac-427f-a8da-bec575ab8b3e") : secret "dns-default-metrics-tls" not found Jul 15 12:24:03.073055 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.073019 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-nnjv2"] Jul 15 12:24:03.092118 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.092089 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.097550 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.097526 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-nnjv2"] Jul 15 12:24:03.098949 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.098903 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 12:24:03.099183 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.098900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sgmm5\"" Jul 15 12:24:03.099303 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.099217 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 12:24:03.219449 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.219371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4492539b-db0b-4b2d-98fa-18d543de1458-nginx-conf\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.219613 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.219498 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.321299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.320752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4492539b-db0b-4b2d-98fa-18d543de1458-nginx-conf\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.321299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.320857 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.321299 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:03.320988 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:24:03.321299 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:03.321051 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert podName:4492539b-db0b-4b2d-98fa-18d543de1458 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:03.821033523 +0000 UTC m=+44.371793033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-nnjv2" (UID: "4492539b-db0b-4b2d-98fa-18d543de1458") : secret "networking-console-plugin-cert" not found Jul 15 12:24:03.321787 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.321765 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4492539b-db0b-4b2d-98fa-18d543de1458-nginx-conf\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.624506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.624464 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:03.625042 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:03.624661 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 12:24:03.625042 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:03.624739 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls podName:39e72a12-f0d8-4463-9c9e-5c48da5717c7 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:05.624718936 +0000 UTC m=+46.175478461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls") pod "cluster-samples-operator-65db858894-g22p5" (UID: "39e72a12-f0d8-4463-9c9e-5c48da5717c7") : secret "samples-operator-tls" not found Jul 15 12:24:03.826159 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:03.826125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:03.826312 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:03.826291 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:24:03.826384 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:03.826370 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert podName:4492539b-db0b-4b2d-98fa-18d543de1458 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:04.826349616 +0000 UTC m=+45.377109126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-nnjv2" (UID: "4492539b-db0b-4b2d-98fa-18d543de1458") : secret "networking-console-plugin-cert" not found Jul 15 12:24:04.530928 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:04.530894 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:04.530949 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:04.530967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.531055 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.531057 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.531070 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:08.531054322 +0000 UTC m=+49.081813836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : configmap references non-existent config key: service-ca.crt Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.531092 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:08.531082129 +0000 UTC m=+49.081841639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : secret "router-metrics-certs-default" not found Jul 15 12:24:04.531137 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.531104 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls podName:a523205d-1b6d-4987-a42a-d3de5c5481de nodeName:}" failed. No retries permitted until 2026-07-15 12:24:08.531098601 +0000 UTC m=+49.081858111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wpxm9" (UID: "a523205d-1b6d-4987-a42a-d3de5c5481de") : secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:04.837779 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:04.837285 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:04.837779 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.837587 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:24:04.837779 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:04.837651 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert podName:4492539b-db0b-4b2d-98fa-18d543de1458 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:06.837636686 +0000 UTC m=+47.388396195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-nnjv2" (UID: "4492539b-db0b-4b2d-98fa-18d543de1458") : secret "networking-console-plugin-cert" not found Jul 15 12:24:05.644225 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:05.644147 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:05.644369 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:05.644304 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 12:24:05.644369 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:05.644367 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls podName:39e72a12-f0d8-4463-9c9e-5c48da5717c7 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:09.644352002 +0000 UTC m=+50.195111512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls") pod "cluster-samples-operator-65db858894-g22p5" (UID: "39e72a12-f0d8-4463-9c9e-5c48da5717c7") : secret "samples-operator-tls" not found Jul 15 12:24:06.853677 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:06.853644 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:06.854022 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:06.853832 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:24:06.854022 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:06.853907 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert podName:4492539b-db0b-4b2d-98fa-18d543de1458 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:10.853886095 +0000 UTC m=+51.404645619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-nnjv2" (UID: "4492539b-db0b-4b2d-98fa-18d543de1458") : secret "networking-console-plugin-cert" not found Jul 15 12:24:06.870511 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:06.870484 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-xg6rc"] Jul 15 12:24:06.876094 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:06.876022 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be01789_354a_4605_ab90_c7813324c30d.slice/crio-e6461def10b16afcdb7a23a2857cc679a512fca074b8702461368b36d0188495 WatchSource:0}: Error finding container e6461def10b16afcdb7a23a2857cc679a512fca074b8702461368b36d0188495: Status 404 returned error can't find the container with id e6461def10b16afcdb7a23a2857cc679a512fca074b8702461368b36d0188495 Jul 15 12:24:07.353260 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.353224 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" event={"ID":"19bf4b2b-2ac1-4b62-a476-df4f617e74f8","Type":"ContainerStarted","Data":"4186ed0390bbad7b1c7a953654f546805b8ad0928519ce8b954a10736362fcc6"} Jul 15 12:24:07.355173 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.355150 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" event={"ID":"7eadd2a4-e5a7-4ad9-9689-83437e981ffd","Type":"ContainerStarted","Data":"859bd125e207654b9aed623fe729920c772c8e8929ad2a6ab414a6a117017266"} Jul 15 12:24:07.358154 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.357006 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" event={"ID":"d3e9dae6-5079-4972-b9b2-c5d72c7351ad","Type":"ContainerStarted","Data":"cf19dcd95daea840c11456478f9b20b50032b336e8530846dd7974394fabcb5f"} Jul 15 12:24:07.358997 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.358972 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" event={"ID":"97c63a62-8a37-4495-90c9-0944d7fdb759","Type":"ContainerStarted","Data":"381f65815d7400e93593afde4bade78e824699284794d427744469103a3aed10"} Jul 15 12:24:07.360553 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.360526 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" event={"ID":"1e360f0c-fc2b-4915-92ab-61a3c4b63e71","Type":"ContainerStarted","Data":"f4af06f563610b0fa26793a601b77511972c7fc95602e9a731ceea7e42a87a54"} Jul 15 12:24:07.361674 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.361655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xg6rc" event={"ID":"7be01789-354a-4605-ab90-c7813324c30d","Type":"ContainerStarted","Data":"e6461def10b16afcdb7a23a2857cc679a512fca074b8702461368b36d0188495"} Jul 15 12:24:07.375134 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.375089 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" podStartSLOduration=2.004942499 podStartE2EDuration="7.375073058s" totalImagesPullingTime="5.370130559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:07.374418931 +0000 UTC m=+47.925178465" watchObservedRunningTime="2026-07-15 12:24:07.375073058 +0000 UTC m=+47.925832592" Jul 15 12:24:07.398449 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.398407 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" podStartSLOduration=2.155062789 podStartE2EDuration="7.398389859s" totalImagesPullingTime="5.24332707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:07.397039569 +0000 UTC m=+47.947799107" watchObservedRunningTime="2026-07-15 12:24:07.398389859 +0000 UTC m=+47.949149394" Jul 15 12:24:07.428243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.428188 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" podStartSLOduration=2.163769386 podStartE2EDuration="7.428172346s" totalImagesPullingTime="5.26440296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:07.426076949 +0000 UTC m=+47.976836484" watchObservedRunningTime="2026-07-15 12:24:07.428172346 +0000 UTC m=+47.978931879" Jul 15 12:24:07.480886 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.480649 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-nst9b" podStartSLOduration=2.322654819 podStartE2EDuration="7.480631248s" totalImagesPullingTime="5.157976429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:07.479798262 +0000 UTC m=+48.030557795" watchObservedRunningTime="2026-07-15 12:24:07.480631248 +0000 UTC m=+48.031390780" Jul 15 12:24:07.481379 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:07.481323 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-p5s6c" podStartSLOduration=2.26091476 podStartE2EDuration="7.481312756s" totalImagesPullingTime="5.220397996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:07.456825218 +0000 UTC m=+48.007584755" watchObservedRunningTime="2026-07-15 12:24:07.481312756 +0000 UTC m=+48.032072289" Jul 15 12:24:08.211168 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.209718 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vfr2c"] Jul 15 12:24:08.249880 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.249850 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vfr2c"] Jul 15 12:24:08.250043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.249984 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.253461 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.253435 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 12:24:08.253574 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.253435 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 12:24:08.253645 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.253626 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x7vt8\"" Jul 15 12:24:08.266883 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.266860 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j44n6\" (UniqueName: \"kubernetes.io/projected/60110441-30ee-4331-9881-17af64ac1a99-kube-api-access-j44n6\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.266982 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.266906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/60110441-30ee-4331-9881-17af64ac1a99-crio-socket\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.266982 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.266932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.266982 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.266956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.267132 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.267032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/60110441-30ee-4331-9881-17af64ac1a99-data-volume\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368048 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368021 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/60110441-30ee-4331-9881-17af64ac1a99-data-volume\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368195 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368120 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j44n6\" (UniqueName: \"kubernetes.io/projected/60110441-30ee-4331-9881-17af64ac1a99-kube-api-access-j44n6\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368195 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368174 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/60110441-30ee-4331-9881-17af64ac1a99-crio-socket\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368310 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368242 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/60110441-30ee-4331-9881-17af64ac1a99-crio-socket\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368368 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368368 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368353 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/60110441-30ee-4331-9881-17af64ac1a99-data-volume\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368466 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.368376 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.368466 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.368424 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:24:08.368561 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.368499 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:24:08.368561 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.368505 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:08.868484249 +0000 UTC m=+49.419243760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : configmap "kube-rbac-proxy" not found Jul 15 12:24:08.368561 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.368553 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:08.868538025 +0000 UTC m=+49.419297541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : secret "insights-runtime-extractor-tls" not found Jul 15 12:24:08.378001 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.377977 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j44n6\" (UniqueName: \"kubernetes.io/projected/60110441-30ee-4331-9881-17af64ac1a99-kube-api-access-j44n6\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.570813 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.570721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:08.570813 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.570763 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:08.571083 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.570897 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:08.571083 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.570931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:08.571083 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.570975 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls podName:a523205d-1b6d-4987-a42a-d3de5c5481de nodeName:}" failed. No retries permitted until 2026-07-15 12:24:16.570951697 +0000 UTC m=+57.121711225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wpxm9" (UID: "a523205d-1b6d-4987-a42a-d3de5c5481de") : secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:08.571083 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.570980 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 12:24:08.571083 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.571029 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:16.571014248 +0000 UTC m=+57.121773758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : configmap references non-existent config key: service-ca.crt Jul 15 12:24:08.571083 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.571053 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:16.571043211 +0000 UTC m=+57.121802726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : secret "router-metrics-certs-default" not found Jul 15 12:24:08.873540 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.873450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.873540 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:08.873493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:08.873766 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.873599 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:24:08.873766 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.873626 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:24:08.873766 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.873683 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:09.873657922 +0000 UTC m=+50.424417449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : configmap "kube-rbac-proxy" not found Jul 15 12:24:08.873766 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:08.873704 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:09.873694929 +0000 UTC m=+50.424454448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : secret "insights-runtime-extractor-tls" not found Jul 15 12:24:09.022018 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.021930 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r"] Jul 15 12:24:09.049267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.049237 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r"] Jul 15 12:24:09.049456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.049369 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" Jul 15 12:24:09.052681 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.052641 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-52dzk\"" Jul 15 12:24:09.052681 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.052670 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 12:24:09.052967 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.052726 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:09.075691 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.075663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpqp5\" (UniqueName: \"kubernetes.io/projected/daf7b2a0-74e0-4a82-9131-0f262df3912b-kube-api-access-tpqp5\") pod \"migrator-7f59b54f6c-84h6r\" (UID: \"daf7b2a0-74e0-4a82-9131-0f262df3912b\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" Jul 15 12:24:09.176703 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.176625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpqp5\" (UniqueName: \"kubernetes.io/projected/daf7b2a0-74e0-4a82-9131-0f262df3912b-kube-api-access-tpqp5\") pod \"migrator-7f59b54f6c-84h6r\" (UID: \"daf7b2a0-74e0-4a82-9131-0f262df3912b\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" Jul 15 12:24:09.185781 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.185756 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpqp5\" (UniqueName: \"kubernetes.io/projected/daf7b2a0-74e0-4a82-9131-0f262df3912b-kube-api-access-tpqp5\") pod \"migrator-7f59b54f6c-84h6r\" (UID: \"daf7b2a0-74e0-4a82-9131-0f262df3912b\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" Jul 15 12:24:09.361399 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.361362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" Jul 15 12:24:09.414067 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.414044 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nr9tq_489c2f9f-1652-4042-9366-dbb76fac0840/dns-node-resolver/0.log" Jul 15 12:24:09.681722 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.681691 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:09.681871 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:09.681853 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 12:24:09.681926 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:09.681918 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls podName:39e72a12-f0d8-4463-9c9e-5c48da5717c7 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:17.68190262 +0000 UTC m=+58.232662129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls") pod "cluster-samples-operator-65db858894-g22p5" (UID: "39e72a12-f0d8-4463-9c9e-5c48da5717c7") : secret "samples-operator-tls" not found Jul 15 12:24:09.688367 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.688344 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r"] Jul 15 12:24:09.690352 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:09.690329 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf7b2a0_74e0_4a82_9131_0f262df3912b.slice/crio-d510ccc1dbf2228c597d26c9010335e198676a21deec78218c47aeb66b3d0172 WatchSource:0}: Error finding container d510ccc1dbf2228c597d26c9010335e198676a21deec78218c47aeb66b3d0172: Status 404 returned error can't find the container with id d510ccc1dbf2228c597d26c9010335e198676a21deec78218c47aeb66b3d0172 Jul 15 12:24:09.883925 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.883849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:09.883925 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:09.883887 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:09.884115 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:09.883969 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:24:09.884115 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:09.883985 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:24:09.884115 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:09.884032 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:11.884017512 +0000 UTC m=+52.434777022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : configmap "kube-rbac-proxy" not found Jul 15 12:24:09.884115 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:09.884107 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:11.884099808 +0000 UTC m=+52.434859317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : secret "insights-runtime-extractor-tls" not found Jul 15 12:24:10.373893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.373846 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/0.log" Jul 15 12:24:10.374346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.373911 2559 generic.go:356] "Generic (PLEG): container finished" podID="7be01789-354a-4605-ab90-c7813324c30d" containerID="b94e17ec72b2493260f77a18701d2918858f2522f261b987e014d7f43aa7e084" exitCode=255 Jul 15 12:24:10.374346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.373992 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xg6rc" event={"ID":"7be01789-354a-4605-ab90-c7813324c30d","Type":"ContainerDied","Data":"b94e17ec72b2493260f77a18701d2918858f2522f261b987e014d7f43aa7e084"} Jul 15 12:24:10.374346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.374266 2559 scope.go:117] "RemoveContainer" containerID="b94e17ec72b2493260f77a18701d2918858f2522f261b987e014d7f43aa7e084" Jul 15 12:24:10.375492 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.375461 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" event={"ID":"daf7b2a0-74e0-4a82-9131-0f262df3912b","Type":"ContainerStarted","Data":"d510ccc1dbf2228c597d26c9010335e198676a21deec78218c47aeb66b3d0172"} Jul 15 12:24:10.608276 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.608249 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xksn9_90572c3f-6cb5-4c41-8b12-222f3041b25d/node-ca/0.log" Jul 15 12:24:10.690980 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.690948 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:10.691135 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.691093 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 12:24:10.691135 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.691114 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f9cf7dd9-nqzbp: secret "image-registry-tls" not found Jul 15 12:24:10.691197 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.691167 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls podName:e076e1be-3bea-4ee7-a70c-ee8f10a4bf98 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:26.691153777 +0000 UTC m=+67.241913288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls") pod "image-registry-64f9cf7dd9-nqzbp" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98") : secret "image-registry-tls" not found Jul 15 12:24:10.792477 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.792397 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:10.792714 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.792694 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 12:24:10.792779 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.792772 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls podName:62c2d3bc-8cac-427f-a8da-bec575ab8b3e nodeName:}" failed. No retries permitted until 2026-07-15 12:24:26.792751751 +0000 UTC m=+67.343511267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls") pod "dns-default-jm9h6" (UID: "62c2d3bc-8cac-427f-a8da-bec575ab8b3e") : secret "dns-default-metrics-tls" not found Jul 15 12:24:10.893131 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:10.893099 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:10.893310 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.893252 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:24:10.893398 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:10.893326 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert podName:4492539b-db0b-4b2d-98fa-18d543de1458 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:18.893310175 +0000 UTC m=+59.444069689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-nnjv2" (UID: "4492539b-db0b-4b2d-98fa-18d543de1458") : secret "networking-console-plugin-cert" not found Jul 15 12:24:11.007482 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.007449 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8tw9h_f35b6a0c-30d5-486c-a98b-adb08a528ea7/serve-healthcheck-canary/0.log" Jul 15 12:24:11.379512 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.379491 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:24:11.379928 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.379913 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/0.log" Jul 15 12:24:11.380010 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.379953 2559 generic.go:356] "Generic (PLEG): container finished" podID="7be01789-354a-4605-ab90-c7813324c30d" containerID="15430f2dcecf536e9cd114b8b9b8e3eb8e10e1d63a081311dc54234842f66b43" exitCode=255 Jul 15 12:24:11.380066 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.380045 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xg6rc" event={"ID":"7be01789-354a-4605-ab90-c7813324c30d","Type":"ContainerDied","Data":"15430f2dcecf536e9cd114b8b9b8e3eb8e10e1d63a081311dc54234842f66b43"} Jul 15 12:24:11.380140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.380114 2559 scope.go:117] "RemoveContainer" containerID="b94e17ec72b2493260f77a18701d2918858f2522f261b987e014d7f43aa7e084" Jul 15 12:24:11.380346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.380328 2559 scope.go:117] "RemoveContainer" containerID="15430f2dcecf536e9cd114b8b9b8e3eb8e10e1d63a081311dc54234842f66b43" Jul 15 12:24:11.380521 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:11.380500 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-xg6rc_openshift-console-operator(7be01789-354a-4605-ab90-c7813324c30d)\"" pod="openshift-console-operator/console-operator-7866497494-xg6rc" podUID="7be01789-354a-4605-ab90-c7813324c30d" Jul 15 12:24:11.381618 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.381599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" event={"ID":"daf7b2a0-74e0-4a82-9131-0f262df3912b","Type":"ContainerStarted","Data":"f93c1e1b86d2528b0995651cf7a4017c44510ba4188c5c57bdf61f6d94a8ee20"} Jul 15 12:24:11.407428 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.407405 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-45vqp"] Jul 15 12:24:11.439875 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.439847 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-45vqp"] Jul 15 12:24:11.439983 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.439956 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.443063 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.443044 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 15 12:24:11.443180 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.443163 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 15 12:24:11.443346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.443324 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 15 12:24:11.443402 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.443375 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 15 12:24:11.443493 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.443479 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cb74l\"" Jul 15 12:24:11.499237 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.499214 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpp55\" (UniqueName: \"kubernetes.io/projected/5305d439-908b-4a1a-9e1b-1a8b42d2637f-kube-api-access-vpp55\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.499525 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.499486 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5305d439-908b-4a1a-9e1b-1a8b42d2637f-signing-key\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.499696 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.499546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5305d439-908b-4a1a-9e1b-1a8b42d2637f-signing-cabundle\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.600596 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.600522 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5305d439-908b-4a1a-9e1b-1a8b42d2637f-signing-key\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.600596 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.600566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5305d439-908b-4a1a-9e1b-1a8b42d2637f-signing-cabundle\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.600820 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.600773 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpp55\" (UniqueName: \"kubernetes.io/projected/5305d439-908b-4a1a-9e1b-1a8b42d2637f-kube-api-access-vpp55\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.601158 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.601133 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5305d439-908b-4a1a-9e1b-1a8b42d2637f-signing-cabundle\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.602844 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.602797 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5305d439-908b-4a1a-9e1b-1a8b42d2637f-signing-key\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.610364 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.610341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpp55\" (UniqueName: \"kubernetes.io/projected/5305d439-908b-4a1a-9e1b-1a8b42d2637f-kube-api-access-vpp55\") pod \"service-ca-79f68b9f4f-45vqp\" (UID: \"5305d439-908b-4a1a-9e1b-1a8b42d2637f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.748875 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.748837 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" Jul 15 12:24:11.890060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.889991 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-45vqp"] Jul 15 12:24:11.892775 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:11.892750 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5305d439_908b_4a1a_9e1b_1a8b42d2637f.slice/crio-efa659657b753388ce596527aa2bd7ea4df9e56207a6eef168777ccc7dd6545f WatchSource:0}: Error finding container efa659657b753388ce596527aa2bd7ea4df9e56207a6eef168777ccc7dd6545f: Status 404 returned error can't find the container with id efa659657b753388ce596527aa2bd7ea4df9e56207a6eef168777ccc7dd6545f Jul 15 12:24:11.903658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.903636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:11.903760 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.903671 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:11.903832 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:11.903771 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:24:11.903832 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:11.903825 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:15.903813102 +0000 UTC m=+56.454572624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : secret "insights-runtime-extractor-tls" not found Jul 15 12:24:11.904138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:11.904122 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60110441-30ee-4331-9881-17af64ac1a99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:12.386697 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.386660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" event={"ID":"5305d439-908b-4a1a-9e1b-1a8b42d2637f","Type":"ContainerStarted","Data":"a45485ee465114e678c21583f9f1cb989cf4bbd219e4338478a5421c6ea21613"} Jul 15 12:24:12.387128 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.386721 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" event={"ID":"5305d439-908b-4a1a-9e1b-1a8b42d2637f","Type":"ContainerStarted","Data":"efa659657b753388ce596527aa2bd7ea4df9e56207a6eef168777ccc7dd6545f"} Jul 15 12:24:12.388214 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.388192 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:24:12.388526 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.388510 2559 scope.go:117] "RemoveContainer" containerID="15430f2dcecf536e9cd114b8b9b8e3eb8e10e1d63a081311dc54234842f66b43" Jul 15 12:24:12.388699 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:12.388679 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-xg6rc_openshift-console-operator(7be01789-354a-4605-ab90-c7813324c30d)\"" pod="openshift-console-operator/console-operator-7866497494-xg6rc" podUID="7be01789-354a-4605-ab90-c7813324c30d" Jul 15 12:24:12.389864 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.389843 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" event={"ID":"daf7b2a0-74e0-4a82-9131-0f262df3912b","Type":"ContainerStarted","Data":"f2230dda487fc8832a6259e07000a466d04c4b1e7a57f66680d456b319cd10f8"} Jul 15 12:24:12.454299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.454257 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-45vqp" podStartSLOduration=1.454242777 podStartE2EDuration="1.454242777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:12.42926559 +0000 UTC m=+52.980025133" watchObservedRunningTime="2026-07-15 12:24:12.454242777 +0000 UTC m=+53.005002351" Jul 15 12:24:12.455142 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.455112 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-84h6r" podStartSLOduration=2.867809039 podStartE2EDuration="4.45510198s" totalImagesPullingTime="1.587292941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:12.454401535 +0000 UTC m=+53.005161153" watchObservedRunningTime="2026-07-15 12:24:12.45510198 +0000 UTC m=+53.005861512" Jul 15 12:24:12.702753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.702682 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:12.702753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:12.702742 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:13.394023 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:13.393992 2559 scope.go:117] "RemoveContainer" containerID="15430f2dcecf536e9cd114b8b9b8e3eb8e10e1d63a081311dc54234842f66b43" Jul 15 12:24:13.394491 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:13.394181 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-xg6rc_openshift-console-operator(7be01789-354a-4605-ab90-c7813324c30d)\"" pod="openshift-console-operator/console-operator-7866497494-xg6rc" podUID="7be01789-354a-4605-ab90-c7813324c30d" Jul 15 12:24:15.939557 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:15.939527 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:15.940076 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:15.939670 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:24:15.940076 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:15.939746 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls podName:60110441-30ee-4331-9881-17af64ac1a99 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:23.93973121 +0000 UTC m=+64.490490722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vfr2c" (UID: "60110441-30ee-4331-9881-17af64ac1a99") : secret "insights-runtime-extractor-tls" not found Jul 15 12:24:16.646702 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:16.646665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:16.646896 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:16.646746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:16.646896 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:16.646775 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:16.646896 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:16.646870 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:32.646848002 +0000 UTC m=+73.197607535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : configmap references non-existent config key: service-ca.crt Jul 15 12:24:16.647062 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:16.646907 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:16.647062 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:16.646911 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 12:24:16.647062 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:16.646977 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls podName:a523205d-1b6d-4987-a42a-d3de5c5481de nodeName:}" failed. No retries permitted until 2026-07-15 12:24:32.646963527 +0000 UTC m=+73.197723037 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wpxm9" (UID: "a523205d-1b6d-4987-a42a-d3de5c5481de") : secret "cluster-monitoring-operator-tls" not found Jul 15 12:24:16.647062 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:16.647026 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs podName:1effb694-29c1-4b54-8822-604b929a4284 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:32.647008413 +0000 UTC m=+73.197767927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs") pod "router-default-79c7ddc6b-857qm" (UID: "1effb694-29c1-4b54-8822-604b929a4284") : secret "router-metrics-certs-default" not found Jul 15 12:24:17.756655 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:17.756622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:17.759137 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:17.759102 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/39e72a12-f0d8-4463-9c9e-5c48da5717c7-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-g22p5\" (UID: \"39e72a12-f0d8-4463-9c9e-5c48da5717c7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:18.015965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:18.015883 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" Jul 15 12:24:18.145733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:18.145709 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5"] Jul 15 12:24:18.413962 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:18.413888 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" event={"ID":"39e72a12-f0d8-4463-9c9e-5c48da5717c7","Type":"ContainerStarted","Data":"ae1322272c32db008e0924f552711623e7c2883dfb68969cf28946b2dc109fcb"} Jul 15 12:24:18.969551 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:18.969514 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:18.970017 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:18.969686 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:24:18.970017 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:24:18.969765 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert podName:4492539b-db0b-4b2d-98fa-18d543de1458 nodeName:}" failed. No retries permitted until 2026-07-15 12:24:34.969744358 +0000 UTC m=+75.520503874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-nnjv2" (UID: "4492539b-db0b-4b2d-98fa-18d543de1458") : secret "networking-console-plugin-cert" not found Jul 15 12:24:20.287501 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:20.287472 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swsfr" Jul 15 12:24:20.422390 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:20.422355 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" event={"ID":"39e72a12-f0d8-4463-9c9e-5c48da5717c7","Type":"ContainerStarted","Data":"0e440b77cb45f076e8d6eba91f2560694fdb5ce4058f1cb32228aa5ce843b646"} Jul 15 12:24:20.422547 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:20.422408 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" event={"ID":"39e72a12-f0d8-4463-9c9e-5c48da5717c7","Type":"ContainerStarted","Data":"cfa1718637dc79b5291f9248584efe4abf684637e7c210b1407f349977ec10de"} Jul 15 12:24:20.444234 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:20.444181 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-g22p5" podStartSLOduration=18.92464691 podStartE2EDuration="20.444166213s" totalImagesPullingTime="1.519519303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:20.442649644 +0000 UTC m=+60.993409175" watchObservedRunningTime="2026-07-15 12:24:20.444166213 +0000 UTC m=+60.994925771" Jul 15 12:24:24.016563 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:24.016497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:24.018920 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:24.018879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60110441-30ee-4331-9881-17af64ac1a99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vfr2c\" (UID: \"60110441-30ee-4331-9881-17af64ac1a99\") " pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:24.163130 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:24.163097 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x7vt8\"" Jul 15 12:24:24.171033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:24.171016 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vfr2c" Jul 15 12:24:24.299204 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:24.299125 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vfr2c"] Jul 15 12:24:24.302330 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:24.302306 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60110441_30ee_4331_9881_17af64ac1a99.slice/crio-36c40803be49cc6f0e17c7a7e37e455995403b9fc745bb831c38c01822ba426b WatchSource:0}: Error finding container 36c40803be49cc6f0e17c7a7e37e455995403b9fc745bb831c38c01822ba426b: Status 404 returned error can't find the container with id 36c40803be49cc6f0e17c7a7e37e455995403b9fc745bb831c38c01822ba426b Jul 15 12:24:24.438506 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:24.438472 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vfr2c" event={"ID":"60110441-30ee-4331-9881-17af64ac1a99","Type":"ContainerStarted","Data":"36c40803be49cc6f0e17c7a7e37e455995403b9fc745bb831c38c01822ba426b"} Jul 15 12:24:25.732283 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.732242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:24:25.735461 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.735432 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 12:24:25.745625 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.745592 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09106fe3-c171-4e91-b958-ba44ff83852e-metrics-certs\") pod \"network-metrics-daemon-dcp89\" (UID: \"09106fe3-c171-4e91-b958-ba44ff83852e\") " pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:24:25.832977 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.832947 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:24:25.833112 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.833003 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:24:25.835838 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.835796 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzl9b\" (UniqueName: \"kubernetes.io/projected/080bb3f7-f2f2-4417-845b-519ee121d45e-kube-api-access-bzl9b\") pod \"network-check-target-mbn5n\" (UID: \"080bb3f7-f2f2-4417-845b-519ee121d45e\") " pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:24:25.836723 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.836700 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 12:24:25.845633 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:25.845610 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741e3629-2b48-4694-a392-189d31991a8f-original-pull-secret\") pod \"global-pull-secret-syncer-5qqkj\" (UID: \"741e3629-2b48-4694-a392-189d31991a8f\") " pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:24:26.016644 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.016573 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qqkj" Jul 15 12:24:26.027997 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.027972 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x2twz\"" Jul 15 12:24:26.034945 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.034923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcp89" Jul 15 12:24:26.035526 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.035503 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qvxmk\"" Jul 15 12:24:26.042580 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.042430 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:24:26.176121 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.175976 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qqkj"] Jul 15 12:24:26.184066 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:26.183500 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741e3629_2b48_4694_a392_189d31991a8f.slice/crio-376b68abf7ddeeea6724f469c6ae19ddd363875e2c065a1d5fd9d9bda1cfc071 WatchSource:0}: Error finding container 376b68abf7ddeeea6724f469c6ae19ddd363875e2c065a1d5fd9d9bda1cfc071: Status 404 returned error can't find the container with id 376b68abf7ddeeea6724f469c6ae19ddd363875e2c065a1d5fd9d9bda1cfc071 Jul 15 12:24:26.204693 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:26.204667 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09106fe3_c171_4e91_b958_ba44ff83852e.slice/crio-dbe567ba179d490d417e45c67dd144771b49763f2ed560f2ab790e72dc1b6d3d WatchSource:0}: Error finding container dbe567ba179d490d417e45c67dd144771b49763f2ed560f2ab790e72dc1b6d3d: Status 404 returned error can't find the container with id dbe567ba179d490d417e45c67dd144771b49763f2ed560f2ab790e72dc1b6d3d Jul 15 12:24:26.207569 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.207525 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dcp89"] Jul 15 12:24:26.220652 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.220627 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mbn5n"] Jul 15 12:24:26.224837 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:26.224797 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080bb3f7_f2f2_4417_845b_519ee121d45e.slice/crio-4181e8bb7971603235f72f313c72c775cbe049984b0da8244fcadc887c712039 WatchSource:0}: Error finding container 4181e8bb7971603235f72f313c72c775cbe049984b0da8244fcadc887c712039: Status 404 returned error can't find the container with id 4181e8bb7971603235f72f313c72c775cbe049984b0da8244fcadc887c712039 Jul 15 12:24:26.446839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.446792 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mbn5n" event={"ID":"080bb3f7-f2f2-4417-845b-519ee121d45e","Type":"ContainerStarted","Data":"ee9bc26ec660d69f1d192f9b78ae5e19ee0d23db2d367454b3efbec061166dca"} Jul 15 12:24:26.447099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.446857 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mbn5n" event={"ID":"080bb3f7-f2f2-4417-845b-519ee121d45e","Type":"ContainerStarted","Data":"4181e8bb7971603235f72f313c72c775cbe049984b0da8244fcadc887c712039"} Jul 15 12:24:26.447099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.447002 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:24:26.448045 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.448019 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcp89" event={"ID":"09106fe3-c171-4e91-b958-ba44ff83852e","Type":"ContainerStarted","Data":"dbe567ba179d490d417e45c67dd144771b49763f2ed560f2ab790e72dc1b6d3d"} Jul 15 12:24:26.449118 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.449098 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qqkj" event={"ID":"741e3629-2b48-4694-a392-189d31991a8f","Type":"ContainerStarted","Data":"376b68abf7ddeeea6724f469c6ae19ddd363875e2c065a1d5fd9d9bda1cfc071"} Jul 15 12:24:26.466848 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.466779 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mbn5n" podStartSLOduration=66.466765338 podStartE2EDuration="1m6.466765338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:26.465586114 +0000 UTC m=+67.016345665" watchObservedRunningTime="2026-07-15 12:24:26.466765338 +0000 UTC m=+67.017524869" Jul 15 12:24:26.742634 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.742599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:26.745329 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.745306 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"image-registry-64f9cf7dd9-nqzbp\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:26.843442 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.843408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:26.846012 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.845986 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d3bc-8cac-427f-a8da-bec575ab8b3e-metrics-tls\") pod \"dns-default-jm9h6\" (UID: \"62c2d3bc-8cac-427f-a8da-bec575ab8b3e\") " pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:26.975354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.975323 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4zvck\"" Jul 15 12:24:26.981903 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.981876 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:26.995730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:26.995506 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ggrzt\"" Jul 15 12:24:27.003173 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:27.003152 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:28.104703 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:28.104672 2559 scope.go:117] "RemoveContainer" containerID="15430f2dcecf536e9cd114b8b9b8e3eb8e10e1d63a081311dc54234842f66b43" Jul 15 12:24:29.869324 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:29.869288 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f9cf7dd9-nqzbp"] Jul 15 12:24:29.872890 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:29.872860 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode076e1be_3bea_4ee7_a70c_ee8f10a4bf98.slice/crio-a5cbbbf45918a36b25f35005ff419800444bc27d5506e14f8f3870f0d9b57c6b WatchSource:0}: Error finding container a5cbbbf45918a36b25f35005ff419800444bc27d5506e14f8f3870f0d9b57c6b: Status 404 returned error can't find the container with id a5cbbbf45918a36b25f35005ff419800444bc27d5506e14f8f3870f0d9b57c6b Jul 15 12:24:29.887260 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:29.887239 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jm9h6"] Jul 15 12:24:30.051003 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:30.050971 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c2d3bc_8cac_427f_a8da_bec575ab8b3e.slice/crio-5ef8fd677feca9a3793c92d75106b58c0e5fc5de7ff332604729f0c1cd09838e WatchSource:0}: Error finding container 5ef8fd677feca9a3793c92d75106b58c0e5fc5de7ff332604729f0c1cd09838e: Status 404 returned error can't find the container with id 5ef8fd677feca9a3793c92d75106b58c0e5fc5de7ff332604729f0c1cd09838e Jul 15 12:24:30.468176 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.468144 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" event={"ID":"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98","Type":"ContainerStarted","Data":"a5cbbbf45918a36b25f35005ff419800444bc27d5506e14f8f3870f0d9b57c6b"} Jul 15 12:24:30.469634 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.469562 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jm9h6" event={"ID":"62c2d3bc-8cac-427f-a8da-bec575ab8b3e","Type":"ContainerStarted","Data":"5ef8fd677feca9a3793c92d75106b58c0e5fc5de7ff332604729f0c1cd09838e"} Jul 15 12:24:30.471936 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.471919 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:24:30.472032 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.471976 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xg6rc" event={"ID":"7be01789-354a-4605-ab90-c7813324c30d","Type":"ContainerStarted","Data":"030881534b1c87818b58316846e1ec3a04038db8751828efe80e7faca8c20f81"} Jul 15 12:24:30.472238 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.472218 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:30.482932 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.482888 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-xg6rc" Jul 15 12:24:30.498413 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:30.498371 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-xg6rc" podStartSLOduration=27.804820635 podStartE2EDuration="30.498354845s" totalImagesPullingTime="2.69353421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:30.497867199 +0000 UTC m=+71.048626750" watchObservedRunningTime="2026-07-15 12:24:30.498354845 +0000 UTC m=+71.049114374" Jul 15 12:24:31.480141 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.479662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcp89" event={"ID":"09106fe3-c171-4e91-b958-ba44ff83852e","Type":"ContainerStarted","Data":"2f23325f717ef6ffc46e59b064e31dbc82a9667444da304c775c673f557c5381"} Jul 15 12:24:31.480141 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.479730 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcp89" event={"ID":"09106fe3-c171-4e91-b958-ba44ff83852e","Type":"ContainerStarted","Data":"4bfa5a887c61d1a2d1a1320313821c53a332310d88e39832b43aaea768baf9eb"} Jul 15 12:24:31.482087 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.482059 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" event={"ID":"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98","Type":"ContainerStarted","Data":"8090e32a1fdae69f05e308127b505e5bf3c5726fdc7803a40feaf5d3e11b6118"} Jul 15 12:24:31.482230 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.482214 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:31.491823 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.491702 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vfr2c" event={"ID":"60110441-30ee-4331-9881-17af64ac1a99","Type":"ContainerStarted","Data":"765c7d047959e5a8be05a9889348e55e3902503f15f706993292847f8b1df8d0"} Jul 15 12:24:31.517107 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.515760 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dcp89" podStartSLOduration=67.030191326 podStartE2EDuration="1m11.515741274s" totalImagesPullingTime="4.485549948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:31.515146602 +0000 UTC m=+72.065906136" watchObservedRunningTime="2026-07-15 12:24:31.515741274 +0000 UTC m=+72.066500806" Jul 15 12:24:31.538980 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:31.538387 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" podStartSLOduration=70.538374868 podStartE2EDuration="1m10.538374868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:31.538285133 +0000 UTC m=+72.089044694" watchObservedRunningTime="2026-07-15 12:24:31.538374868 +0000 UTC m=+72.089134401" Jul 15 12:24:32.695820 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.695766 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:32.696289 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.695856 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:32.696289 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.695885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:32.696418 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.696397 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1effb694-29c1-4b54-8822-604b929a4284-service-ca-bundle\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:32.698845 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.698821 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a523205d-1b6d-4987-a42a-d3de5c5481de-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wpxm9\" (UID: \"a523205d-1b6d-4987-a42a-d3de5c5481de\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:32.698929 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.698841 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1effb694-29c1-4b54-8822-604b929a4284-metrics-certs\") pod \"router-default-79c7ddc6b-857qm\" (UID: \"1effb694-29c1-4b54-8822-604b929a4284\") " pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:32.866213 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.866177 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6r7rb\"" Jul 15 12:24:32.870127 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.870103 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zkfkj\"" Jul 15 12:24:32.872121 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.872096 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" Jul 15 12:24:32.878199 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:32.878178 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:33.391901 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.391867 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-djq2j"] Jul 15 12:24:33.398849 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.398826 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:33.403108 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.402597 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 12:24:33.403108 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.403080 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wbd8m\"" Jul 15 12:24:33.404298 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.404263 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 12:24:33.420763 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.420740 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-djq2j"] Jul 15 12:24:33.503519 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.503484 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frrj2\" (UniqueName: \"kubernetes.io/projected/d765379d-99a6-4043-95cb-2fe71731a3ef-kube-api-access-frrj2\") pod \"downloads-68c9cbb497-djq2j\" (UID: \"d765379d-99a6-4043-95cb-2fe71731a3ef\") " pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:33.604075 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.604044 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frrj2\" (UniqueName: \"kubernetes.io/projected/d765379d-99a6-4043-95cb-2fe71731a3ef-kube-api-access-frrj2\") pod \"downloads-68c9cbb497-djq2j\" (UID: \"d765379d-99a6-4043-95cb-2fe71731a3ef\") " pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:33.614432 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.614408 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frrj2\" (UniqueName: \"kubernetes.io/projected/d765379d-99a6-4043-95cb-2fe71731a3ef-kube-api-access-frrj2\") pod \"downloads-68c9cbb497-djq2j\" (UID: \"d765379d-99a6-4043-95cb-2fe71731a3ef\") " pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:33.710923 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:33.710892 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:35.017030 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:35.016995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:35.019368 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:35.019341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4492539b-db0b-4b2d-98fa-18d543de1458-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-nnjv2\" (UID: \"4492539b-db0b-4b2d-98fa-18d543de1458\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:35.208904 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:35.208874 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sgmm5\"" Jul 15 12:24:35.214618 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:35.214584 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" Jul 15 12:24:35.844229 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:35.844204 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-nnjv2"] Jul 15 12:24:35.847371 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:35.847332 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4492539b_db0b_4b2d_98fa_18d543de1458.slice/crio-e01b1a7981e5a4acf3d77d4141e387cd5f3115dd5cc31756d4204d62d4ff4e5e WatchSource:0}: Error finding container e01b1a7981e5a4acf3d77d4141e387cd5f3115dd5cc31756d4204d62d4ff4e5e: Status 404 returned error can't find the container with id e01b1a7981e5a4acf3d77d4141e387cd5f3115dd5cc31756d4204d62d4ff4e5e Jul 15 12:24:35.907498 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:35.907455 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-djq2j"] Jul 15 12:24:35.911146 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:35.911114 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd765379d_99a6_4043_95cb_2fe71731a3ef.slice/crio-e1ffb0fd5fc6e871eb79f119eb9ff6d5416ae20d8e5c136bbe194d1c58bc2b6a WatchSource:0}: Error finding container e1ffb0fd5fc6e871eb79f119eb9ff6d5416ae20d8e5c136bbe194d1c58bc2b6a: Status 404 returned error can't find the container with id e1ffb0fd5fc6e871eb79f119eb9ff6d5416ae20d8e5c136bbe194d1c58bc2b6a Jul 15 12:24:36.075381 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.075313 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9"] Jul 15 12:24:36.078191 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:36.078162 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda523205d_1b6d_4987_a42a_d3de5c5481de.slice/crio-239119bcfd70b32fb24c4df3dbadeafcb4819a6c5ae577adb29a2c82a87f174d WatchSource:0}: Error finding container 239119bcfd70b32fb24c4df3dbadeafcb4819a6c5ae577adb29a2c82a87f174d: Status 404 returned error can't find the container with id 239119bcfd70b32fb24c4df3dbadeafcb4819a6c5ae577adb29a2c82a87f174d Jul 15 12:24:36.081029 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.081002 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79c7ddc6b-857qm"] Jul 15 12:24:36.084473 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:36.084445 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1effb694_29c1_4b54_8822_604b929a4284.slice/crio-81309dc8b9ec076726b56fef9263d55f105c77cbc1a1237950593251542ddf05 WatchSource:0}: Error finding container 81309dc8b9ec076726b56fef9263d55f105c77cbc1a1237950593251542ddf05: Status 404 returned error can't find the container with id 81309dc8b9ec076726b56fef9263d55f105c77cbc1a1237950593251542ddf05 Jul 15 12:24:36.519963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.519912 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jm9h6" event={"ID":"62c2d3bc-8cac-427f-a8da-bec575ab8b3e","Type":"ContainerStarted","Data":"cfac41b72be64838a910159e6fd0b0961f44aa9ce20562c1cb292c538093269e"} Jul 15 12:24:36.520133 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.519986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jm9h6" event={"ID":"62c2d3bc-8cac-427f-a8da-bec575ab8b3e","Type":"ContainerStarted","Data":"aff9512447e2d1caf3ea1754d5879aee77c2cfe3bd9a9745bb19d4f9f9b56892"} Jul 15 12:24:36.520133 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.520022 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:36.521485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.521452 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79c7ddc6b-857qm" event={"ID":"1effb694-29c1-4b54-8822-604b929a4284","Type":"ContainerStarted","Data":"443d77705b6d7b3999281d3bb12f540d2e4f792aa59cd0c780a58c7b2f9cecab"} Jul 15 12:24:36.521604 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.521498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79c7ddc6b-857qm" event={"ID":"1effb694-29c1-4b54-8822-604b929a4284","Type":"ContainerStarted","Data":"81309dc8b9ec076726b56fef9263d55f105c77cbc1a1237950593251542ddf05"} Jul 15 12:24:36.522689 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.522661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" event={"ID":"a523205d-1b6d-4987-a42a-d3de5c5481de","Type":"ContainerStarted","Data":"239119bcfd70b32fb24c4df3dbadeafcb4819a6c5ae577adb29a2c82a87f174d"} Jul 15 12:24:36.523758 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.523739 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-djq2j" event={"ID":"d765379d-99a6-4043-95cb-2fe71731a3ef","Type":"ContainerStarted","Data":"e1ffb0fd5fc6e871eb79f119eb9ff6d5416ae20d8e5c136bbe194d1c58bc2b6a"} Jul 15 12:24:36.525632 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.525612 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vfr2c" event={"ID":"60110441-30ee-4331-9881-17af64ac1a99","Type":"ContainerStarted","Data":"a126b54e435d1fba9096e56d43169bb81f9a0cbb2a1d86c6f157a6475caca9e9"} Jul 15 12:24:36.526760 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.526738 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" event={"ID":"4492539b-db0b-4b2d-98fa-18d543de1458","Type":"ContainerStarted","Data":"e01b1a7981e5a4acf3d77d4141e387cd5f3115dd5cc31756d4204d62d4ff4e5e"} Jul 15 12:24:36.528046 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.528025 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qqkj" event={"ID":"741e3629-2b48-4694-a392-189d31991a8f","Type":"ContainerStarted","Data":"649401338ca9b4269b77394e277578466e795b7589ac11086c676502f09110aa"} Jul 15 12:24:36.547622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.547573 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jm9h6" podStartSLOduration=36.977348827 podStartE2EDuration="42.547558632s" totalImagesPullingTime="5.570209805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:36.547110127 +0000 UTC m=+77.097869669" watchObservedRunningTime="2026-07-15 12:24:36.547558632 +0000 UTC m=+77.098318163" Jul 15 12:24:36.617884 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.617830 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79c7ddc6b-857qm" podStartSLOduration=36.617790177 podStartE2EDuration="36.617790177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:36.607561009 +0000 UTC m=+77.158320562" watchObservedRunningTime="2026-07-15 12:24:36.617790177 +0000 UTC m=+77.168549708" Jul 15 12:24:36.618030 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.617954 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5qqkj" podStartSLOduration=66.153386135 podStartE2EDuration="1m15.617947725s" totalImagesPullingTime="9.46456159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:36.573111948 +0000 UTC m=+77.123871480" watchObservedRunningTime="2026-07-15 12:24:36.617947725 +0000 UTC m=+77.168707258" Jul 15 12:24:36.878599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.878491 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:36.882321 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:36.882076 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:37.535096 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:37.535064 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:37.536550 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:37.536470 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79c7ddc6b-857qm" Jul 15 12:24:38.796839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.796788 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cdf7dbdf9-82ktw"] Jul 15 12:24:38.808424 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.808381 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:38.812062 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.811991 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdf7dbdf9-82ktw"] Jul 15 12:24:38.817735 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.817708 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 12:24:38.818714 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.817840 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jhjl8\"" Jul 15 12:24:38.818910 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.817927 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 12:24:38.819111 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.818026 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 12:24:38.819196 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.818089 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 12:24:38.819256 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.818147 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 12:24:38.946565 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.946515 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-service-ca\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:38.946565 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.946568 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-serving-cert\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:38.946878 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.946655 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhgvr\" (UniqueName: \"kubernetes.io/projected/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-kube-api-access-lhgvr\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:38.946878 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.946692 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-config\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:38.946878 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.946730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-oauth-serving-cert\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:38.946878 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:38.946763 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-oauth-config\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.047947 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.047852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-service-ca\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.047947 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.047901 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-serving-cert\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.047947 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.047936 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhgvr\" (UniqueName: \"kubernetes.io/projected/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-kube-api-access-lhgvr\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.048243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.047962 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-config\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.048243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.047983 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-oauth-serving-cert\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.048243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.048004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-oauth-config\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.048776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.048690 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-service-ca\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.048776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.048725 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-config\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.048976 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.048957 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-oauth-serving-cert\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.051369 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.051350 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-oauth-config\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.051480 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.051389 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-serving-cert\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.058299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.058274 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhgvr\" (UniqueName: \"kubernetes.io/projected/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-kube-api-access-lhgvr\") pod \"console-7cdf7dbdf9-82ktw\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:39.122580 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:39.122534 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:41.222775 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.222747 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdf7dbdf9-82ktw"] Jul 15 12:24:41.224026 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:41.223997 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8529fbb_f4c8_4c85_b61c_dcbdad9de036.slice/crio-fbaf9e962213f1b8811855414bc1f21f9764a7d9a720969ccbde08445439ddb3 WatchSource:0}: Error finding container fbaf9e962213f1b8811855414bc1f21f9764a7d9a720969ccbde08445439ddb3: Status 404 returned error can't find the container with id fbaf9e962213f1b8811855414bc1f21f9764a7d9a720969ccbde08445439ddb3 Jul 15 12:24:41.554865 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.554824 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" event={"ID":"a523205d-1b6d-4987-a42a-d3de5c5481de","Type":"ContainerStarted","Data":"ba8e4d85b8ce079f9e604871a28b0acf50c4465ec704cc3c857284c9a7012e19"} Jul 15 12:24:41.556631 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.556599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdf7dbdf9-82ktw" event={"ID":"a8529fbb-f4c8-4c85-b61c-dcbdad9de036","Type":"ContainerStarted","Data":"fbaf9e962213f1b8811855414bc1f21f9764a7d9a720969ccbde08445439ddb3"} Jul 15 12:24:41.559141 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.559099 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vfr2c" event={"ID":"60110441-30ee-4331-9881-17af64ac1a99","Type":"ContainerStarted","Data":"d145d7f92e0c67d2f4ad00e7aa3c1635ffdb3169d52fa01a00c378d8151d8340"} Jul 15 12:24:41.561035 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.561010 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" event={"ID":"4492539b-db0b-4b2d-98fa-18d543de1458","Type":"ContainerStarted","Data":"353e98f87a38737ace5c8d0a24b098bad395aa57b86015b747981d21eea62952"} Jul 15 12:24:41.575868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.575797 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wpxm9" podStartSLOduration=36.582851384 podStartE2EDuration="41.575782809s" totalImagesPullingTime="4.992931425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:41.574155454 +0000 UTC m=+82.124914986" watchObservedRunningTime="2026-07-15 12:24:41.575782809 +0000 UTC m=+82.126542341" Jul 15 12:24:41.597891 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.597830 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vfr2c" podStartSLOduration=17.065885947 podStartE2EDuration="33.597795544s" totalImagesPullingTime="16.531909597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:24:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:41.596093281 +0000 UTC m=+82.146852814" watchObservedRunningTime="2026-07-15 12:24:41.597795544 +0000 UTC m=+82.148555078" Jul 15 12:24:41.618536 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:41.618485 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-nnjv2" podStartSLOduration=33.394117716 podStartE2EDuration="38.618470899s" totalImagesPullingTime="5.224353183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:41.616848709 +0000 UTC m=+82.167608262" watchObservedRunningTime="2026-07-15 12:24:41.618470899 +0000 UTC m=+82.169230498" Jul 15 12:24:44.581595 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:44.581555 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdf7dbdf9-82ktw" event={"ID":"a8529fbb-f4c8-4c85-b61c-dcbdad9de036","Type":"ContainerStarted","Data":"94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3"} Jul 15 12:24:44.603183 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:44.603127 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cdf7dbdf9-82ktw" podStartSLOduration=3.39872159 podStartE2EDuration="6.603111962s" totalImagesPullingTime="3.204390372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:44.602314147 +0000 UTC m=+85.153073700" watchObservedRunningTime="2026-07-15 12:24:44.603111962 +0000 UTC m=+85.153871541" Jul 15 12:24:46.537387 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:46.537357 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jm9h6" Jul 15 12:24:46.987197 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:46.987150 2559 patch_prober.go:28] interesting pod/image-registry-64f9cf7dd9-nqzbp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 12:24:46.987343 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:46.987232 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" podUID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 12:24:49.122699 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:49.122658 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:49.123161 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:49.122726 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:49.124357 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:49.124329 2559 patch_prober.go:28] interesting pod/console-7cdf7dbdf9-82ktw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" start-of-body= Jul 15 12:24:49.124483 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:49.124393 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7cdf7dbdf9-82ktw" podUID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" containerName="console" probeResult="failure" output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" Jul 15 12:24:52.510556 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:52.510525 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:24:55.470197 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:55.470167 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64f9cf7dd9-nqzbp"] Jul 15 12:24:56.635019 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:56.634978 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-djq2j" event={"ID":"d765379d-99a6-4043-95cb-2fe71731a3ef","Type":"ContainerStarted","Data":"9a49d14fbd92b451831f88e3b3b38e7765cbbfc4269bb8a3ed8a7214e7e93a0a"} Jul 15 12:24:56.635498 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:56.635159 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:56.650731 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:56.650708 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-djq2j" Jul 15 12:24:56.658866 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:56.658793 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-djq2j" podStartSLOduration=3.849022625 podStartE2EDuration="23.658774556s" totalImagesPullingTime="19.809751931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:56.656758649 +0000 UTC m=+97.207518227" watchObservedRunningTime="2026-07-15 12:24:56.658774556 +0000 UTC m=+97.209534089" Jul 15 12:24:57.457218 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:57.457186 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mbn5n" Jul 15 12:24:59.127686 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.127654 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:59.131743 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.131720 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:24:59.513862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.513829 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jkk8f"] Jul 15 12:24:59.547711 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.547626 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.554273 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.554252 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 12:24:59.554707 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.554253 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2f47l\"" Jul 15 12:24:59.554707 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.554293 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 12:24:59.555131 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.555114 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 12:24:59.555397 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.555381 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 12:24:59.623036 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623152 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623044 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfs6p\" (UniqueName: \"kubernetes.io/projected/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-kube-api-access-bfs6p\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623152 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623115 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-tls\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623245 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623163 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-sys\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623245 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-root\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623245 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623245 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623236 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-textfile\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623412 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-metrics-client-ca\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.623412 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.623301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-wtmp\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.723884 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.723854 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724004 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.723897 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfs6p\" (UniqueName: \"kubernetes.io/projected/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-kube-api-access-bfs6p\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724004 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.723960 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-tls\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724120 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-sys\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724120 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724057 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-root\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724120 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724107 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-root\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724270 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724148 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-sys\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724270 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724371 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724313 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-textfile\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724461 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-metrics-client-ca\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724643 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724600 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-textfile\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724743 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724644 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-wtmp\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.724839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.724795 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-wtmp\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.725078 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.725033 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.725078 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.725058 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-metrics-client-ca\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.727059 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.727017 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-tls\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.727152 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.727057 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.736681 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.736648 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfs6p\" (UniqueName: \"kubernetes.io/projected/08e0f63a-6f49-4a8e-b7e7-003b23ac63f5-kube-api-access-bfs6p\") pod \"node-exporter-jkk8f\" (UID: \"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5\") " pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.862140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:24:59.862067 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkk8f" Jul 15 12:24:59.897922 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:24:59.897897 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e0f63a_6f49_4a8e_b7e7_003b23ac63f5.slice/crio-49386ed14276e1a2e43c29f515c7e59d5357bf0a0537504f5ef074193d953927 WatchSource:0}: Error finding container 49386ed14276e1a2e43c29f515c7e59d5357bf0a0537504f5ef074193d953927: Status 404 returned error can't find the container with id 49386ed14276e1a2e43c29f515c7e59d5357bf0a0537504f5ef074193d953927 Jul 15 12:25:00.655647 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:00.655607 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk8f" event={"ID":"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5","Type":"ContainerStarted","Data":"49386ed14276e1a2e43c29f515c7e59d5357bf0a0537504f5ef074193d953927"} Jul 15 12:25:01.662914 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:01.662458 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk8f" event={"ID":"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5","Type":"ContainerStarted","Data":"639d0f0881e57da07fd18854cd913fbbdf18180ccb18c77fd3ddab754dab8d49"} Jul 15 12:25:02.669079 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:02.669041 2559 generic.go:356] "Generic (PLEG): container finished" podID="08e0f63a-6f49-4a8e-b7e7-003b23ac63f5" containerID="639d0f0881e57da07fd18854cd913fbbdf18180ccb18c77fd3ddab754dab8d49" exitCode=0 Jul 15 12:25:02.669550 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:02.669138 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk8f" event={"ID":"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5","Type":"ContainerDied","Data":"639d0f0881e57da07fd18854cd913fbbdf18180ccb18c77fd3ddab754dab8d49"} Jul 15 12:25:03.676926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:03.676869 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk8f" event={"ID":"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5","Type":"ContainerStarted","Data":"10325bb225cd766a3f1196f6552210f330f291079774ee83a8f080dfbca259db"} Jul 15 12:25:03.677365 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:03.676959 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk8f" event={"ID":"08e0f63a-6f49-4a8e-b7e7-003b23ac63f5","Type":"ContainerStarted","Data":"5a3b1e884366053792bfb32ff85193f824dd2582d3b383c49ac535e2c97e19f0"} Jul 15 12:25:12.712701 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:12.712673 2559 generic.go:356] "Generic (PLEG): container finished" podID="19bf4b2b-2ac1-4b62-a476-df4f617e74f8" containerID="4186ed0390bbad7b1c7a953654f546805b8ad0928519ce8b954a10736362fcc6" exitCode=0 Jul 15 12:25:12.713051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:12.712740 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" event={"ID":"19bf4b2b-2ac1-4b62-a476-df4f617e74f8","Type":"ContainerDied","Data":"4186ed0390bbad7b1c7a953654f546805b8ad0928519ce8b954a10736362fcc6"} Jul 15 12:25:12.713051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:12.713043 2559 scope.go:117] "RemoveContainer" containerID="4186ed0390bbad7b1c7a953654f546805b8ad0928519ce8b954a10736362fcc6" Jul 15 12:25:12.735725 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:12.735685 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jkk8f" podStartSLOduration=12.087480077 podStartE2EDuration="13.735673454s" totalImagesPullingTime="1.541326682s" totalInitContainerRuntime="106.866695ms" isStatefulPod=true podCreationTimestamp="2026-07-15 12:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:25:03.707914289 +0000 UTC m=+104.258673842" watchObservedRunningTime="2026-07-15 12:25:12.735673454 +0000 UTC m=+113.286432986" Jul 15 12:25:13.718865 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:13.718828 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dp9wg" event={"ID":"19bf4b2b-2ac1-4b62-a476-df4f617e74f8","Type":"ContainerStarted","Data":"44c44737a7f0521b306bf862f9d8add31fcf078904150c747e62916a308173d5"} Jul 15 12:25:14.929152 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:14.929120 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdf7dbdf9-82ktw"] Jul 15 12:25:20.490578 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.490542 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" podUID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" containerName="registry" containerID="cri-o://8090e32a1fdae69f05e308127b505e5bf3c5726fdc7803a40feaf5d3e11b6118" gracePeriod=30 Jul 15 12:25:20.744483 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.744429 2559 generic.go:356] "Generic (PLEG): container finished" podID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" containerID="8090e32a1fdae69f05e308127b505e5bf3c5726fdc7803a40feaf5d3e11b6118" exitCode=0 Jul 15 12:25:20.744606 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.744515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" event={"ID":"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98","Type":"ContainerDied","Data":"8090e32a1fdae69f05e308127b505e5bf3c5726fdc7803a40feaf5d3e11b6118"} Jul 15 12:25:20.744606 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.744574 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" event={"ID":"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98","Type":"ContainerDied","Data":"a5cbbbf45918a36b25f35005ff419800444bc27d5506e14f8f3870f0d9b57c6b"} Jul 15 12:25:20.744606 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.744593 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5cbbbf45918a36b25f35005ff419800444bc27d5506e14f8f3870f0d9b57c6b" Jul 15 12:25:20.761753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.761734 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:25:20.890839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.890815 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-image-registry-private-configuration\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.890951 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.890851 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-bound-sa-token\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.890951 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.890876 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-trusted-ca\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.890951 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.890912 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.890951 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.890938 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-installation-pull-secrets\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.891151 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.891001 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-certificates\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.891151 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.891032 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsh2z\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-kube-api-access-gsh2z\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.891151 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.891088 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-ca-trust-extracted\") pod \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\" (UID: \"e076e1be-3bea-4ee7-a70c-ee8f10a4bf98\") " Jul 15 12:25:20.891420 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.891345 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:25:20.891723 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.891696 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:25:20.893302 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.893277 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:25:20.893461 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.893331 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:25:20.893571 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.893486 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:25:20.893626 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.893565 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-kube-api-access-gsh2z" (OuterVolumeSpecName: "kube-api-access-gsh2z") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "kube-api-access-gsh2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:25:20.893944 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.893921 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:25:20.902033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.902008 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" (UID: "e076e1be-3bea-4ee7-a70c-ee8f10a4bf98"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:25:20.992188 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992168 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-ca-trust-extracted\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992188 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992188 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-image-registry-private-configuration\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992301 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992198 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-bound-sa-token\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992301 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992206 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-trusted-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992301 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992213 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-tls\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992301 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992221 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-installation-pull-secrets\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992301 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992229 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-registry-certificates\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:20.992301 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:20.992237 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsh2z\" (UniqueName: \"kubernetes.io/projected/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98-kube-api-access-gsh2z\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:21.748011 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:21.747980 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f9cf7dd9-nqzbp" Jul 15 12:25:21.772322 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:21.772298 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64f9cf7dd9-nqzbp"] Jul 15 12:25:21.776020 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:21.775998 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64f9cf7dd9-nqzbp"] Jul 15 12:25:22.108972 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:22.108897 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" path="/var/lib/kubelet/pods/e076e1be-3bea-4ee7-a70c-ee8f10a4bf98/volumes" Jul 15 12:25:27.769963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:27.769930 2559 generic.go:356] "Generic (PLEG): container finished" podID="1e360f0c-fc2b-4915-92ab-61a3c4b63e71" containerID="f4af06f563610b0fa26793a601b77511972c7fc95602e9a731ceea7e42a87a54" exitCode=0 Jul 15 12:25:27.770370 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:27.770009 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" event={"ID":"1e360f0c-fc2b-4915-92ab-61a3c4b63e71","Type":"ContainerDied","Data":"f4af06f563610b0fa26793a601b77511972c7fc95602e9a731ceea7e42a87a54"} Jul 15 12:25:27.770370 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:27.770352 2559 scope.go:117] "RemoveContainer" containerID="f4af06f563610b0fa26793a601b77511972c7fc95602e9a731ceea7e42a87a54" Jul 15 12:25:28.775446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:28.775416 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bmp27" event={"ID":"1e360f0c-fc2b-4915-92ab-61a3c4b63e71","Type":"ContainerStarted","Data":"4b368747d77adc35b9cc62f20529c607c87fe80cdee1e1d4b62e7272de37ddf5"} Jul 15 12:25:33.795475 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:33.795438 2559 generic.go:356] "Generic (PLEG): container finished" podID="7eadd2a4-e5a7-4ad9-9689-83437e981ffd" containerID="859bd125e207654b9aed623fe729920c772c8e8929ad2a6ab414a6a117017266" exitCode=0 Jul 15 12:25:33.795913 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:33.795484 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" event={"ID":"7eadd2a4-e5a7-4ad9-9689-83437e981ffd","Type":"ContainerDied","Data":"859bd125e207654b9aed623fe729920c772c8e8929ad2a6ab414a6a117017266"} Jul 15 12:25:33.795913 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:33.795780 2559 scope.go:117] "RemoveContainer" containerID="859bd125e207654b9aed623fe729920c772c8e8929ad2a6ab414a6a117017266" Jul 15 12:25:34.800794 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:34.800756 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qszx8" event={"ID":"7eadd2a4-e5a7-4ad9-9689-83437e981ffd","Type":"ContainerStarted","Data":"df18f9b2908e8a68c0da3d875d838565f1dd4f431adf7fb90b43adadcb865547"} Jul 15 12:25:39.950450 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:39.950410 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cdf7dbdf9-82ktw" podUID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" containerName="console" containerID="cri-o://94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3" gracePeriod=15 Jul 15 12:25:40.254182 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.254153 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:25:40.338051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338016 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-config\") pod \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " Jul 15 12:25:40.338206 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338112 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-service-ca\") pod \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " Jul 15 12:25:40.338206 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338144 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-serving-cert\") pod \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " Jul 15 12:25:40.338206 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338177 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-oauth-config\") pod \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " Jul 15 12:25:40.338359 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338213 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhgvr\" (UniqueName: \"kubernetes.io/projected/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-kube-api-access-lhgvr\") pod \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " Jul 15 12:25:40.338359 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338269 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-oauth-serving-cert\") pod \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\" (UID: \"a8529fbb-f4c8-4c85-b61c-dcbdad9de036\") " Jul 15 12:25:40.338463 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338403 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-config" (OuterVolumeSpecName: "console-config") pod "a8529fbb-f4c8-4c85-b61c-dcbdad9de036" (UID: "a8529fbb-f4c8-4c85-b61c-dcbdad9de036"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:25:40.338524 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338481 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-config\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:40.338603 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338571 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8529fbb-f4c8-4c85-b61c-dcbdad9de036" (UID: "a8529fbb-f4c8-4c85-b61c-dcbdad9de036"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:25:40.338655 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.338634 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8529fbb-f4c8-4c85-b61c-dcbdad9de036" (UID: "a8529fbb-f4c8-4c85-b61c-dcbdad9de036"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:25:40.340439 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.340398 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-kube-api-access-lhgvr" (OuterVolumeSpecName: "kube-api-access-lhgvr") pod "a8529fbb-f4c8-4c85-b61c-dcbdad9de036" (UID: "a8529fbb-f4c8-4c85-b61c-dcbdad9de036"). InnerVolumeSpecName "kube-api-access-lhgvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:25:40.340583 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.340449 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8529fbb-f4c8-4c85-b61c-dcbdad9de036" (UID: "a8529fbb-f4c8-4c85-b61c-dcbdad9de036"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:25:40.340583 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.340473 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8529fbb-f4c8-4c85-b61c-dcbdad9de036" (UID: "a8529fbb-f4c8-4c85-b61c-dcbdad9de036"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:25:40.439748 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.439716 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-oauth-serving-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:40.439748 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.439745 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:40.439748 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.439755 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-serving-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:40.439984 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.439764 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-console-oauth-config\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:40.439984 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.439774 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhgvr\" (UniqueName: \"kubernetes.io/projected/a8529fbb-f4c8-4c85-b61c-dcbdad9de036-kube-api-access-lhgvr\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:25:40.827244 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.827212 2559 generic.go:356] "Generic (PLEG): container finished" podID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" containerID="94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3" exitCode=0 Jul 15 12:25:40.827382 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.827303 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdf7dbdf9-82ktw" event={"ID":"a8529fbb-f4c8-4c85-b61c-dcbdad9de036","Type":"ContainerDied","Data":"94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3"} Jul 15 12:25:40.827382 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.827330 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdf7dbdf9-82ktw" Jul 15 12:25:40.827382 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.827370 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdf7dbdf9-82ktw" event={"ID":"a8529fbb-f4c8-4c85-b61c-dcbdad9de036","Type":"ContainerDied","Data":"fbaf9e962213f1b8811855414bc1f21f9764a7d9a720969ccbde08445439ddb3"} Jul 15 12:25:40.827502 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.827389 2559 scope.go:117] "RemoveContainer" containerID="94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3" Jul 15 12:25:40.836316 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.836293 2559 scope.go:117] "RemoveContainer" containerID="94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3" Jul 15 12:25:40.836626 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:25:40.836607 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3\": container with ID starting with 94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3 not found: ID does not exist" containerID="94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3" Jul 15 12:25:40.836700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.836633 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3"} err="failed to get container status \"94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3\": rpc error: code = NotFound desc = could not find container \"94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3\": container with ID starting with 94525c7b473d6234cda27671ad72f03026f1316a3e384146df1bb2c4ca0638a3 not found: ID does not exist" Jul 15 12:25:40.866553 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.866519 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdf7dbdf9-82ktw"] Jul 15 12:25:40.882296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:40.882263 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cdf7dbdf9-82ktw"] Jul 15 12:25:42.111565 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:25:42.111536 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" path="/var/lib/kubelet/pods/a8529fbb-f4c8-4c85-b61c-dcbdad9de036/volumes" Jul 15 12:26:27.913291 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913262 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6b7dc5db-46qcf"] Jul 15 12:26:27.913676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913528 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" containerName="registry" Jul 15 12:26:27.913676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913541 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" containerName="registry" Jul 15 12:26:27.913676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913557 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" containerName="console" Jul 15 12:26:27.913676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913563 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" containerName="console" Jul 15 12:26:27.913676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913612 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e076e1be-3bea-4ee7-a70c-ee8f10a4bf98" containerName="registry" Jul 15 12:26:27.913676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.913621 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8529fbb-f4c8-4c85-b61c-dcbdad9de036" containerName="console" Jul 15 12:26:27.917837 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.917815 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.921596 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.921567 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 12:26:27.921720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.921616 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 12:26:27.923479 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.923460 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 12:26:27.924820 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.924780 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 12:26:27.924953 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.924936 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jhjl8\"" Jul 15 12:26:27.925353 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.925328 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 12:26:27.929037 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.929020 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 12:26:27.939134 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.939115 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6b7dc5db-46qcf"] Jul 15 12:26:27.972746 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972722 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-serving-cert\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.972855 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972749 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-trusted-ca-bundle\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.972855 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972770 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-console-config\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.972855 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972832 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-oauth-config\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.972963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972858 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-oauth-serving-cert\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.972963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972916 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sjkf\" (UniqueName: \"kubernetes.io/projected/db03df4c-51dc-4fe8-b125-fdac443518b1-kube-api-access-2sjkf\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:27.972963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:27.972950 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-service-ca\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073525 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sjkf\" (UniqueName: \"kubernetes.io/projected/db03df4c-51dc-4fe8-b125-fdac443518b1-kube-api-access-2sjkf\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073667 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073553 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-service-ca\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073667 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073577 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-serving-cert\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073667 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-trusted-ca-bundle\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073667 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-console-config\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073667 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073658 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-oauth-config\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.073895 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.073682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-oauth-serving-cert\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.074333 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.074312 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-service-ca\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.074490 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.074467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-oauth-serving-cert\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.074561 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.074467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-console-config\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.074700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.074676 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-trusted-ca-bundle\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.076179 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.076158 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-serving-cert\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.076259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.076162 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-oauth-config\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.083530 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.083509 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sjkf\" (UniqueName: \"kubernetes.io/projected/db03df4c-51dc-4fe8-b125-fdac443518b1-kube-api-access-2sjkf\") pod \"console-c6b7dc5db-46qcf\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.228601 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.228582 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:28.363888 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:28.363865 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6b7dc5db-46qcf"] Jul 15 12:26:28.365979 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:26:28.365951 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb03df4c_51dc_4fe8_b125_fdac443518b1.slice/crio-2b5231c953761ce605fc0c6120df8798efade6c967a4819855a8d96398e84318 WatchSource:0}: Error finding container 2b5231c953761ce605fc0c6120df8798efade6c967a4819855a8d96398e84318: Status 404 returned error can't find the container with id 2b5231c953761ce605fc0c6120df8798efade6c967a4819855a8d96398e84318 Jul 15 12:26:29.019580 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:29.019547 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6b7dc5db-46qcf" event={"ID":"db03df4c-51dc-4fe8-b125-fdac443518b1","Type":"ContainerStarted","Data":"e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588"} Jul 15 12:26:29.019999 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:29.019597 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6b7dc5db-46qcf" event={"ID":"db03df4c-51dc-4fe8-b125-fdac443518b1","Type":"ContainerStarted","Data":"2b5231c953761ce605fc0c6120df8798efade6c967a4819855a8d96398e84318"} Jul 15 12:26:29.043885 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:29.043821 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6b7dc5db-46qcf" podStartSLOduration=2.043783695 podStartE2EDuration="2.043783695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:29.041894053 +0000 UTC m=+189.592653597" watchObservedRunningTime="2026-07-15 12:26:29.043783695 +0000 UTC m=+189.594543227" Jul 15 12:26:34.659937 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.659901 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6b7dc5db-46qcf"] Jul 15 12:26:34.734743 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.734704 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54975b4c7-f9qjr"] Jul 15 12:26:34.738259 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.738236 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.766989 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.766964 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54975b4c7-f9qjr"] Jul 15 12:26:34.821730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.821695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-trusted-ca-bundle\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.821730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.821736 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-oauth-serving-cert\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.822006 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.821755 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-serving-cert\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.822006 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.821892 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk57b\" (UniqueName: \"kubernetes.io/projected/9572266e-06ef-4f42-b3e3-205953ed2ac7-kube-api-access-nk57b\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.822006 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.821946 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-config\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.822006 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.821993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-service-ca\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.822177 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.822024 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-oauth-config\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923359 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923283 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-service-ca\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923359 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923315 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-oauth-config\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923359 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923343 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-trusted-ca-bundle\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-oauth-serving-cert\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923382 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-serving-cert\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk57b\" (UniqueName: \"kubernetes.io/projected/9572266e-06ef-4f42-b3e3-205953ed2ac7-kube-api-access-nk57b\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.923608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.923430 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-config\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.924244 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.924214 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-oauth-serving-cert\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.924354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.924215 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-service-ca\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.924354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.924272 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-trusted-ca-bundle\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.924354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.924307 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-config\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.925697 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.925676 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-serving-cert\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.925794 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.925761 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-oauth-config\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:34.932912 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:34.932890 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk57b\" (UniqueName: \"kubernetes.io/projected/9572266e-06ef-4f42-b3e3-205953ed2ac7-kube-api-access-nk57b\") pod \"console-54975b4c7-f9qjr\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:35.047582 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:35.047554 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:35.181731 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:35.181708 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54975b4c7-f9qjr"] Jul 15 12:26:35.183762 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:26:35.183735 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9572266e_06ef_4f42_b3e3_205953ed2ac7.slice/crio-664be6f39fdfd11383f6feccc2947d2b241a954006d7fffc6f094f160bb6e623 WatchSource:0}: Error finding container 664be6f39fdfd11383f6feccc2947d2b241a954006d7fffc6f094f160bb6e623: Status 404 returned error can't find the container with id 664be6f39fdfd11383f6feccc2947d2b241a954006d7fffc6f094f160bb6e623 Jul 15 12:26:36.049636 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:36.049597 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54975b4c7-f9qjr" event={"ID":"9572266e-06ef-4f42-b3e3-205953ed2ac7","Type":"ContainerStarted","Data":"1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5"} Jul 15 12:26:36.050013 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:36.049654 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54975b4c7-f9qjr" event={"ID":"9572266e-06ef-4f42-b3e3-205953ed2ac7","Type":"ContainerStarted","Data":"664be6f39fdfd11383f6feccc2947d2b241a954006d7fffc6f094f160bb6e623"} Jul 15 12:26:36.076545 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:36.076496 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54975b4c7-f9qjr" podStartSLOduration=2.076482137 podStartE2EDuration="2.076482137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:36.075680787 +0000 UTC m=+196.626440332" watchObservedRunningTime="2026-07-15 12:26:36.076482137 +0000 UTC m=+196.627241669" Jul 15 12:26:38.229112 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:38.229069 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:26:45.048144 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:45.048103 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:45.048539 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:45.048173 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:45.053163 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:45.053138 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:45.090552 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:45.090516 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:26:59.680115 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:59.680073 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c6b7dc5db-46qcf" podUID="db03df4c-51dc-4fe8-b125-fdac443518b1" containerName="console" containerID="cri-o://e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588" gracePeriod=15 Jul 15 12:26:59.942272 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:26:59.942214 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:27:00.026854 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.026797 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-oauth-serving-cert\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027023 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.026891 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-service-ca\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027023 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.026937 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-trusted-ca-bundle\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027023 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.026966 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-console-config\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027139 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027104 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sjkf\" (UniqueName: \"kubernetes.io/projected/db03df4c-51dc-4fe8-b125-fdac443518b1-kube-api-access-2sjkf\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027186 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027168 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-serving-cert\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027239 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027191 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-oauth-config\") pod \"db03df4c-51dc-4fe8-b125-fdac443518b1\" (UID: \"db03df4c-51dc-4fe8-b125-fdac443518b1\") " Jul 15 12:27:00.027293 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027263 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:27:00.027356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027336 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:27:00.027356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027342 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-console-config" (OuterVolumeSpecName: "console-config") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:27:00.027460 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027364 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:27:00.027518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027493 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-oauth-serving-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.027518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027511 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.027611 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027524 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-trusted-ca-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.027611 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.027537 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db03df4c-51dc-4fe8-b125-fdac443518b1-console-config\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.029243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.029219 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:27:00.029386 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.029296 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db03df4c-51dc-4fe8-b125-fdac443518b1-kube-api-access-2sjkf" (OuterVolumeSpecName: "kube-api-access-2sjkf") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "kube-api-access-2sjkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:00.029386 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.029348 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "db03df4c-51dc-4fe8-b125-fdac443518b1" (UID: "db03df4c-51dc-4fe8-b125-fdac443518b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:27:00.128622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.128591 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sjkf\" (UniqueName: \"kubernetes.io/projected/db03df4c-51dc-4fe8-b125-fdac443518b1-kube-api-access-2sjkf\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.128622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.128617 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-serving-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.128622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.128627 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db03df4c-51dc-4fe8-b125-fdac443518b1-console-oauth-config\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:00.144940 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.144913 2559 generic.go:356] "Generic (PLEG): container finished" podID="db03df4c-51dc-4fe8-b125-fdac443518b1" containerID="e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588" exitCode=0 Jul 15 12:27:00.145074 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.144955 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6b7dc5db-46qcf" Jul 15 12:27:00.145074 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.144995 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6b7dc5db-46qcf" event={"ID":"db03df4c-51dc-4fe8-b125-fdac443518b1","Type":"ContainerDied","Data":"e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588"} Jul 15 12:27:00.145074 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.145045 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6b7dc5db-46qcf" event={"ID":"db03df4c-51dc-4fe8-b125-fdac443518b1","Type":"ContainerDied","Data":"2b5231c953761ce605fc0c6120df8798efade6c967a4819855a8d96398e84318"} Jul 15 12:27:00.145074 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.145065 2559 scope.go:117] "RemoveContainer" containerID="e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588" Jul 15 12:27:00.153044 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.153016 2559 scope.go:117] "RemoveContainer" containerID="e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588" Jul 15 12:27:00.153263 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:27:00.153233 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588\": container with ID starting with e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588 not found: ID does not exist" containerID="e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588" Jul 15 12:27:00.153320 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.153271 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588"} err="failed to get container status \"e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588\": rpc error: code = NotFound desc = could not find container \"e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588\": container with ID starting with e5e1964a11a6fe25fc033f6cf5ab6a03b6ea439fab104f5a684a01e0ebffd588 not found: ID does not exist" Jul 15 12:27:00.168688 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.168664 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6b7dc5db-46qcf"] Jul 15 12:27:00.177317 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:00.177296 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c6b7dc5db-46qcf"] Jul 15 12:27:02.108065 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:02.108031 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db03df4c-51dc-4fe8-b125-fdac443518b1" path="/var/lib/kubelet/pods/db03df4c-51dc-4fe8-b125-fdac443518b1/volumes" Jul 15 12:27:31.230576 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.230541 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs"] Jul 15 12:27:31.231090 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.230865 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db03df4c-51dc-4fe8-b125-fdac443518b1" containerName="console" Jul 15 12:27:31.231090 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.230877 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="db03df4c-51dc-4fe8-b125-fdac443518b1" containerName="console" Jul 15 12:27:31.231090 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.230938 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="db03df4c-51dc-4fe8-b125-fdac443518b1" containerName="console" Jul 15 12:27:31.235192 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.235174 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.239937 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.239916 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkr9x\"" Jul 15 12:27:31.241445 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.241385 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:27:31.258712 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.258682 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:27:31.260478 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.260459 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs"] Jul 15 12:27:31.272336 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.272313 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.272429 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.272348 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpgck\" (UniqueName: \"kubernetes.io/projected/5a06f993-b0e1-4a80-9e78-94e809f7518a-kube-api-access-lpgck\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.272429 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.272415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.373127 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.373082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.373329 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.373139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgck\" (UniqueName: \"kubernetes.io/projected/5a06f993-b0e1-4a80-9e78-94e809f7518a-kube-api-access-lpgck\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.373329 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.373215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.373471 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.373458 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.373575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.373554 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.383693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.383665 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpgck\" (UniqueName: \"kubernetes.io/projected/5a06f993-b0e1-4a80-9e78-94e809f7518a-kube-api-access-lpgck\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.544376 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.544291 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:31.674926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:31.674866 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs"] Jul 15 12:27:31.677181 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:27:31.677154 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a06f993_b0e1_4a80_9e78_94e809f7518a.slice/crio-d8840c0a71dab847bbba58b93919d215d1b144a0e65f5709ca9fe86089588d86 WatchSource:0}: Error finding container d8840c0a71dab847bbba58b93919d215d1b144a0e65f5709ca9fe86089588d86: Status 404 returned error can't find the container with id d8840c0a71dab847bbba58b93919d215d1b144a0e65f5709ca9fe86089588d86 Jul 15 12:27:32.266933 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:32.266896 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" event={"ID":"5a06f993-b0e1-4a80-9e78-94e809f7518a","Type":"ContainerStarted","Data":"d8840c0a71dab847bbba58b93919d215d1b144a0e65f5709ca9fe86089588d86"} Jul 15 12:27:39.298504 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:39.298466 2559 generic.go:356] "Generic (PLEG): container finished" podID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerID="c12252fd109fe07cee8479b6ee4b4f3a9936f4d811a353c120468196de264d30" exitCode=0 Jul 15 12:27:39.298886 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:39.298564 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" event={"ID":"5a06f993-b0e1-4a80-9e78-94e809f7518a","Type":"ContainerDied","Data":"c12252fd109fe07cee8479b6ee4b4f3a9936f4d811a353c120468196de264d30"} Jul 15 12:27:42.311910 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:42.311827 2559 generic.go:356] "Generic (PLEG): container finished" podID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerID="c9a505448eb7c4cf1feadafdb04ce2550034bc6dfc72f2c2fa844ee4da6dfbbf" exitCode=0 Jul 15 12:27:42.311910 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:42.311897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" event={"ID":"5a06f993-b0e1-4a80-9e78-94e809f7518a","Type":"ContainerDied","Data":"c9a505448eb7c4cf1feadafdb04ce2550034bc6dfc72f2c2fa844ee4da6dfbbf"} Jul 15 12:27:49.345434 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:49.345334 2559 generic.go:356] "Generic (PLEG): container finished" podID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerID="7d0dbcc45ada0e7fbfa7c78c415cadc23a877bf13f705d8967fa0f1bc9d35bbe" exitCode=0 Jul 15 12:27:49.345796 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:49.345422 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" event={"ID":"5a06f993-b0e1-4a80-9e78-94e809f7518a","Type":"ContainerDied","Data":"7d0dbcc45ada0e7fbfa7c78c415cadc23a877bf13f705d8967fa0f1bc9d35bbe"} Jul 15 12:27:50.479719 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.479694 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:50.528190 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.528162 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-bundle\") pod \"5a06f993-b0e1-4a80-9e78-94e809f7518a\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " Jul 15 12:27:50.528324 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.528209 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpgck\" (UniqueName: \"kubernetes.io/projected/5a06f993-b0e1-4a80-9e78-94e809f7518a-kube-api-access-lpgck\") pod \"5a06f993-b0e1-4a80-9e78-94e809f7518a\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " Jul 15 12:27:50.528324 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.528250 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-util\") pod \"5a06f993-b0e1-4a80-9e78-94e809f7518a\" (UID: \"5a06f993-b0e1-4a80-9e78-94e809f7518a\") " Jul 15 12:27:50.528915 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.528888 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-bundle" (OuterVolumeSpecName: "bundle") pod "5a06f993-b0e1-4a80-9e78-94e809f7518a" (UID: "5a06f993-b0e1-4a80-9e78-94e809f7518a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:50.530420 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.530393 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a06f993-b0e1-4a80-9e78-94e809f7518a-kube-api-access-lpgck" (OuterVolumeSpecName: "kube-api-access-lpgck") pod "5a06f993-b0e1-4a80-9e78-94e809f7518a" (UID: "5a06f993-b0e1-4a80-9e78-94e809f7518a"). InnerVolumeSpecName "kube-api-access-lpgck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:50.532489 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.532468 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-util" (OuterVolumeSpecName: "util") pod "5a06f993-b0e1-4a80-9e78-94e809f7518a" (UID: "5a06f993-b0e1-4a80-9e78-94e809f7518a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:50.628865 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.628763 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpgck\" (UniqueName: \"kubernetes.io/projected/5a06f993-b0e1-4a80-9e78-94e809f7518a-kube-api-access-lpgck\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:50.628865 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.628791 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:50.628865 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:50.628824 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a06f993-b0e1-4a80-9e78-94e809f7518a-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:27:51.354742 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:51.354710 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" event={"ID":"5a06f993-b0e1-4a80-9e78-94e809f7518a","Type":"ContainerDied","Data":"d8840c0a71dab847bbba58b93919d215d1b144a0e65f5709ca9fe86089588d86"} Jul 15 12:27:51.354742 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:51.354741 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpphs" Jul 15 12:27:51.354953 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:51.354754 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8840c0a71dab847bbba58b93919d215d1b144a0e65f5709ca9fe86089588d86" Jul 15 12:27:59.520199 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520164 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx"] Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520451 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="util" Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520461 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="util" Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520480 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="pull" Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520485 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="pull" Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520494 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="extract" Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520499 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="extract" Jul 15 12:27:59.520599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.520543 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a06f993-b0e1-4a80-9e78-94e809f7518a" containerName="extract" Jul 15 12:27:59.525589 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.525573 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.529430 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.529405 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lgbjt\"" Jul 15 12:27:59.529543 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.529464 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 12:27:59.529708 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.529693 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:27:59.541759 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.541737 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx"] Jul 15 12:27:59.605245 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.605214 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b79a3363-2f51-42ae-b957-2db274c19aaf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zwszx\" (UID: \"b79a3363-2f51-42ae-b957-2db274c19aaf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.605366 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.605292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbslr\" (UniqueName: \"kubernetes.io/projected/b79a3363-2f51-42ae-b957-2db274c19aaf-kube-api-access-cbslr\") pod \"cert-manager-operator-controller-manager-55c444bf86-zwszx\" (UID: \"b79a3363-2f51-42ae-b957-2db274c19aaf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.706095 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.706066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbslr\" (UniqueName: \"kubernetes.io/projected/b79a3363-2f51-42ae-b957-2db274c19aaf-kube-api-access-cbslr\") pod \"cert-manager-operator-controller-manager-55c444bf86-zwszx\" (UID: \"b79a3363-2f51-42ae-b957-2db274c19aaf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.706215 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.706122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b79a3363-2f51-42ae-b957-2db274c19aaf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zwszx\" (UID: \"b79a3363-2f51-42ae-b957-2db274c19aaf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.706478 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.706449 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b79a3363-2f51-42ae-b957-2db274c19aaf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zwszx\" (UID: \"b79a3363-2f51-42ae-b957-2db274c19aaf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.721029 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.721004 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbslr\" (UniqueName: \"kubernetes.io/projected/b79a3363-2f51-42ae-b957-2db274c19aaf-kube-api-access-cbslr\") pod \"cert-manager-operator-controller-manager-55c444bf86-zwszx\" (UID: \"b79a3363-2f51-42ae-b957-2db274c19aaf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.835207 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.835151 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" Jul 15 12:27:59.972778 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:27:59.972754 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx"] Jul 15 12:27:59.974327 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:27:59.974306 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79a3363_2f51_42ae_b957_2db274c19aaf.slice/crio-37c97d83c63fabb522467ca0797824a71aa05ebc440eca07c6663385b970b4b3 WatchSource:0}: Error finding container 37c97d83c63fabb522467ca0797824a71aa05ebc440eca07c6663385b970b4b3: Status 404 returned error can't find the container with id 37c97d83c63fabb522467ca0797824a71aa05ebc440eca07c6663385b970b4b3 Jul 15 12:28:00.398232 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:00.398197 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" event={"ID":"b79a3363-2f51-42ae-b957-2db274c19aaf","Type":"ContainerStarted","Data":"37c97d83c63fabb522467ca0797824a71aa05ebc440eca07c6663385b970b4b3"} Jul 15 12:28:03.415129 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:03.415039 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" event={"ID":"b79a3363-2f51-42ae-b957-2db274c19aaf","Type":"ContainerStarted","Data":"32cc99296464cdc47a3341d079f713bdd8349344e5752e5c63b5cb480df7ca33"} Jul 15 12:28:03.489704 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:03.489647 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zwszx" podStartSLOduration=1.33340407 podStartE2EDuration="4.48963158s" totalImagesPullingTime="3.15622751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:03.489017303 +0000 UTC m=+284.039776981" watchObservedRunningTime="2026-07-15 12:28:03.48963158 +0000 UTC m=+284.040391112" Jul 15 12:28:04.420627 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.420595 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7"] Jul 15 12:28:04.424009 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.423992 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.450853 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.450828 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:28:04.450954 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.450890 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:28:04.451756 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.451739 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkr9x\"" Jul 15 12:28:04.488974 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.488949 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7"] Jul 15 12:28:04.549229 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.549207 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfj9v\" (UniqueName: \"kubernetes.io/projected/c6fce285-be62-484d-9be6-d10ea74c4dad-kube-api-access-sfj9v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.549373 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.549254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.549373 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.549344 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.649990 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.649963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.650122 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.650018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.650122 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.650050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfj9v\" (UniqueName: \"kubernetes.io/projected/c6fce285-be62-484d-9be6-d10ea74c4dad-kube-api-access-sfj9v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.650362 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.650342 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.650407 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.650374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.732555 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.732530 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfj9v\" (UniqueName: \"kubernetes.io/projected/c6fce285-be62-484d-9be6-d10ea74c4dad-kube-api-access-sfj9v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.732685 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.732641 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:04.950465 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:04.950430 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7"] Jul 15 12:28:04.952862 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:28:04.952832 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fce285_be62_484d_9be6_d10ea74c4dad.slice/crio-22d122b39f1175fa3ae74fd91be96c132cf40f04a5c9f4712f2cabe23514f144 WatchSource:0}: Error finding container 22d122b39f1175fa3ae74fd91be96c132cf40f04a5c9f4712f2cabe23514f144: Status 404 returned error can't find the container with id 22d122b39f1175fa3ae74fd91be96c132cf40f04a5c9f4712f2cabe23514f144 Jul 15 12:28:05.424518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:05.424442 2559 generic.go:356] "Generic (PLEG): container finished" podID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerID="79a317082fe4394fd389992009d5713d225160326eead062e8bde2330287760c" exitCode=0 Jul 15 12:28:05.424518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:05.424500 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" event={"ID":"c6fce285-be62-484d-9be6-d10ea74c4dad","Type":"ContainerDied","Data":"79a317082fe4394fd389992009d5713d225160326eead062e8bde2330287760c"} Jul 15 12:28:05.424928 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:05.424562 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" event={"ID":"c6fce285-be62-484d-9be6-d10ea74c4dad","Type":"ContainerStarted","Data":"22d122b39f1175fa3ae74fd91be96c132cf40f04a5c9f4712f2cabe23514f144"} Jul 15 12:28:07.437168 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:07.437139 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" event={"ID":"c6fce285-be62-484d-9be6-d10ea74c4dad","Type":"ContainerStarted","Data":"7af57d6be281c2476d540a8c04b04c28ef1dcea5ab7d0c9636a1909b368f4c78"} Jul 15 12:28:07.488308 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:28:07.488278 2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fce285_be62_484d_9be6_d10ea74c4dad.slice/crio-conmon-7af57d6be281c2476d540a8c04b04c28ef1dcea5ab7d0c9636a1909b368f4c78.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fce285_be62_484d_9be6_d10ea74c4dad.slice/crio-7af57d6be281c2476d540a8c04b04c28ef1dcea5ab7d0c9636a1909b368f4c78.scope\": RecentStats: unable to find data in memory cache]" Jul 15 12:28:08.442861 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:08.442823 2559 generic.go:356] "Generic (PLEG): container finished" podID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerID="7af57d6be281c2476d540a8c04b04c28ef1dcea5ab7d0c9636a1909b368f4c78" exitCode=0 Jul 15 12:28:08.443225 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:08.442899 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" event={"ID":"c6fce285-be62-484d-9be6-d10ea74c4dad","Type":"ContainerDied","Data":"7af57d6be281c2476d540a8c04b04c28ef1dcea5ab7d0c9636a1909b368f4c78"} Jul 15 12:28:09.449013 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:09.448983 2559 generic.go:356] "Generic (PLEG): container finished" podID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerID="65bccb72811c4432b43db09c9c400a824b8264e82a004ba1fed51ac003a1ecc3" exitCode=0 Jul 15 12:28:09.449399 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:09.449027 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" event={"ID":"c6fce285-be62-484d-9be6-d10ea74c4dad","Type":"ContainerDied","Data":"65bccb72811c4432b43db09c9c400a824b8264e82a004ba1fed51ac003a1ecc3"} Jul 15 12:28:10.574595 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.574570 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:10.693030 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.693003 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-util\") pod \"c6fce285-be62-484d-9be6-d10ea74c4dad\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " Jul 15 12:28:10.693166 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.693037 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfj9v\" (UniqueName: \"kubernetes.io/projected/c6fce285-be62-484d-9be6-d10ea74c4dad-kube-api-access-sfj9v\") pod \"c6fce285-be62-484d-9be6-d10ea74c4dad\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " Jul 15 12:28:10.693166 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.693059 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-bundle\") pod \"c6fce285-be62-484d-9be6-d10ea74c4dad\" (UID: \"c6fce285-be62-484d-9be6-d10ea74c4dad\") " Jul 15 12:28:10.693485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.693460 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-bundle" (OuterVolumeSpecName: "bundle") pod "c6fce285-be62-484d-9be6-d10ea74c4dad" (UID: "c6fce285-be62-484d-9be6-d10ea74c4dad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:28:10.695148 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.695123 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fce285-be62-484d-9be6-d10ea74c4dad-kube-api-access-sfj9v" (OuterVolumeSpecName: "kube-api-access-sfj9v") pod "c6fce285-be62-484d-9be6-d10ea74c4dad" (UID: "c6fce285-be62-484d-9be6-d10ea74c4dad"). InnerVolumeSpecName "kube-api-access-sfj9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:28:10.698002 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.697982 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-util" (OuterVolumeSpecName: "util") pod "c6fce285-be62-484d-9be6-d10ea74c4dad" (UID: "c6fce285-be62-484d-9be6-d10ea74c4dad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:28:10.794485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.794442 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:28:10.794485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.794482 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfj9v\" (UniqueName: \"kubernetes.io/projected/c6fce285-be62-484d-9be6-d10ea74c4dad-kube-api-access-sfj9v\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:28:10.794485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:10.794492 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6fce285-be62-484d-9be6-d10ea74c4dad-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:28:11.459206 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:11.459178 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" event={"ID":"c6fce285-be62-484d-9be6-d10ea74c4dad","Type":"ContainerDied","Data":"22d122b39f1175fa3ae74fd91be96c132cf40f04a5c9f4712f2cabe23514f144"} Jul 15 12:28:11.459363 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:11.459221 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22d122b39f1175fa3ae74fd91be96c132cf40f04a5c9f4712f2cabe23514f144" Jul 15 12:28:11.459363 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:11.459222 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftxhx7" Jul 15 12:28:16.141663 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.141621 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx"] Jul 15 12:28:16.142309 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142284 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="extract" Jul 15 12:28:16.142396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142313 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="extract" Jul 15 12:28:16.142396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142346 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="pull" Jul 15 12:28:16.142396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142356 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="pull" Jul 15 12:28:16.142396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142384 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="util" Jul 15 12:28:16.142396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142392 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="util" Jul 15 12:28:16.142626 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.142532 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fce285-be62-484d-9be6-d10ea74c4dad" containerName="extract" Jul 15 12:28:16.146105 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.146083 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.151514 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.151490 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5tb9t\"" Jul 15 12:28:16.151846 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.151826 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:28:16.152982 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.152963 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 12:28:16.160192 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.160172 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx"] Jul 15 12:28:16.233189 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.233155 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27155296-5ccf-4287-9c78-87f59c7585c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zfbvx\" (UID: \"27155296-5ccf-4287-9c78-87f59c7585c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.233369 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.233205 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kspml\" (UniqueName: \"kubernetes.io/projected/27155296-5ccf-4287-9c78-87f59c7585c6-kube-api-access-kspml\") pod \"openshift-lws-operator-fd8ccff4c-zfbvx\" (UID: \"27155296-5ccf-4287-9c78-87f59c7585c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.334083 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.334044 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27155296-5ccf-4287-9c78-87f59c7585c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zfbvx\" (UID: \"27155296-5ccf-4287-9c78-87f59c7585c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.334252 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.334094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kspml\" (UniqueName: \"kubernetes.io/projected/27155296-5ccf-4287-9c78-87f59c7585c6-kube-api-access-kspml\") pod \"openshift-lws-operator-fd8ccff4c-zfbvx\" (UID: \"27155296-5ccf-4287-9c78-87f59c7585c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.334474 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.334450 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27155296-5ccf-4287-9c78-87f59c7585c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zfbvx\" (UID: \"27155296-5ccf-4287-9c78-87f59c7585c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.345715 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.345692 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kspml\" (UniqueName: \"kubernetes.io/projected/27155296-5ccf-4287-9c78-87f59c7585c6-kube-api-access-kspml\") pod \"openshift-lws-operator-fd8ccff4c-zfbvx\" (UID: \"27155296-5ccf-4287-9c78-87f59c7585c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.465628 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.465594 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" Jul 15 12:28:16.606432 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:16.606395 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx"] Jul 15 12:28:16.609327 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:28:16.609293 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27155296_5ccf_4287_9c78_87f59c7585c6.slice/crio-0beb1eed2782096b4338618f93faaa2947027968ffb8e5c65f581bc01cc54af6 WatchSource:0}: Error finding container 0beb1eed2782096b4338618f93faaa2947027968ffb8e5c65f581bc01cc54af6: Status 404 returned error can't find the container with id 0beb1eed2782096b4338618f93faaa2947027968ffb8e5c65f581bc01cc54af6 Jul 15 12:28:17.488142 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.488101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" event={"ID":"27155296-5ccf-4287-9c78-87f59c7585c6","Type":"ContainerStarted","Data":"0beb1eed2782096b4338618f93faaa2947027968ffb8e5c65f581bc01cc54af6"} Jul 15 12:28:17.604746 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.604710 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mddcc"] Jul 15 12:28:17.608161 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.608134 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.635842 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.635812 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-vs4xj\"" Jul 15 12:28:17.636305 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.636281 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 12:28:17.641703 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.641686 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 12:28:17.647409 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.647383 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mddcc"] Jul 15 12:28:17.745545 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.745392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef93d08a-267e-4fdb-b2bb-5c5094c8741f-bound-sa-token\") pod \"cert-manager-55b4d954d5-mddcc\" (UID: \"ef93d08a-267e-4fdb-b2bb-5c5094c8741f\") " pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.745545 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.745472 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vllsh\" (UniqueName: \"kubernetes.io/projected/ef93d08a-267e-4fdb-b2bb-5c5094c8741f-kube-api-access-vllsh\") pod \"cert-manager-55b4d954d5-mddcc\" (UID: \"ef93d08a-267e-4fdb-b2bb-5c5094c8741f\") " pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.846421 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.846383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vllsh\" (UniqueName: \"kubernetes.io/projected/ef93d08a-267e-4fdb-b2bb-5c5094c8741f-kube-api-access-vllsh\") pod \"cert-manager-55b4d954d5-mddcc\" (UID: \"ef93d08a-267e-4fdb-b2bb-5c5094c8741f\") " pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.846690 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.846479 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef93d08a-267e-4fdb-b2bb-5c5094c8741f-bound-sa-token\") pod \"cert-manager-55b4d954d5-mddcc\" (UID: \"ef93d08a-267e-4fdb-b2bb-5c5094c8741f\") " pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.859631 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.859604 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef93d08a-267e-4fdb-b2bb-5c5094c8741f-bound-sa-token\") pod \"cert-manager-55b4d954d5-mddcc\" (UID: \"ef93d08a-267e-4fdb-b2bb-5c5094c8741f\") " pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.859776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.859693 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vllsh\" (UniqueName: \"kubernetes.io/projected/ef93d08a-267e-4fdb-b2bb-5c5094c8741f-kube-api-access-vllsh\") pod \"cert-manager-55b4d954d5-mddcc\" (UID: \"ef93d08a-267e-4fdb-b2bb-5c5094c8741f\") " pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:17.918878 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:17.918835 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mddcc" Jul 15 12:28:18.501263 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:18.501227 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mddcc"] Jul 15 12:28:18.503384 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:28:18.503356 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef93d08a_267e_4fdb_b2bb_5c5094c8741f.slice/crio-2aa345586b1932d673d89b2886787b20c28f60601dfec96950d2c60d06214569 WatchSource:0}: Error finding container 2aa345586b1932d673d89b2886787b20c28f60601dfec96950d2c60d06214569: Status 404 returned error can't find the container with id 2aa345586b1932d673d89b2886787b20c28f60601dfec96950d2c60d06214569 Jul 15 12:28:19.499076 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:19.499042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" event={"ID":"27155296-5ccf-4287-9c78-87f59c7585c6","Type":"ContainerStarted","Data":"8aeaccafd5f83aa03da80ceccc161baa8c141d8254abaef08feba003ed1fdc48"} Jul 15 12:28:19.500272 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:19.500250 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mddcc" event={"ID":"ef93d08a-267e-4fdb-b2bb-5c5094c8741f","Type":"ContainerStarted","Data":"2aa345586b1932d673d89b2886787b20c28f60601dfec96950d2c60d06214569"} Jul 15 12:28:19.537869 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:19.537824 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zfbvx" podStartSLOduration=1.358522121 podStartE2EDuration="3.53779597s" totalImagesPullingTime="2.179273849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:19.53573593 +0000 UTC m=+300.086495463" watchObservedRunningTime="2026-07-15 12:28:19.53779597 +0000 UTC m=+300.088555502" Jul 15 12:28:19.892389 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:19.892312 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:28:19.893753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:19.893730 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:28:19.911412 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:19.911390 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 12:28:21.958459 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:21.958426 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q"] Jul 15 12:28:21.961781 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:21.961760 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:21.965889 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:21.965787 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:28:21.965889 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:21.965828 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:28:21.967324 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:21.967303 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkr9x\"" Jul 15 12:28:21.980169 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:21.980144 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q"] Jul 15 12:28:22.080509 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.080477 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.080509 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.080517 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.080720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.080590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c5xk\" (UniqueName: \"kubernetes.io/projected/aefe7681-32b1-42c7-9abe-48f40d58eac1-kube-api-access-5c5xk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.181929 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.181895 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.182111 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.181941 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.182111 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.181986 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c5xk\" (UniqueName: \"kubernetes.io/projected/aefe7681-32b1-42c7-9abe-48f40d58eac1-kube-api-access-5c5xk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.182398 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.182369 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.182484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.182373 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.192396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.192367 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c5xk\" (UniqueName: \"kubernetes.io/projected/aefe7681-32b1-42c7-9abe-48f40d58eac1-kube-api-access-5c5xk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.272957 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.272929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:22.847286 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:22.847265 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q"] Jul 15 12:28:22.848365 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:28:22.848344 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefe7681_32b1_42c7_9abe_48f40d58eac1.slice/crio-9a4a35c9a15a8049786c646cb8c498b56fc53dafb3aed0ff223bfce7b027c65f WatchSource:0}: Error finding container 9a4a35c9a15a8049786c646cb8c498b56fc53dafb3aed0ff223bfce7b027c65f: Status 404 returned error can't find the container with id 9a4a35c9a15a8049786c646cb8c498b56fc53dafb3aed0ff223bfce7b027c65f Jul 15 12:28:23.519977 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:23.519950 2559 generic.go:356] "Generic (PLEG): container finished" podID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerID="53eaea4089ef26fa73ca55e7ba80e6d1869760e2ca5f929169e5398bcda5c4e6" exitCode=0 Jul 15 12:28:23.520356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:23.520040 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" event={"ID":"aefe7681-32b1-42c7-9abe-48f40d58eac1","Type":"ContainerDied","Data":"53eaea4089ef26fa73ca55e7ba80e6d1869760e2ca5f929169e5398bcda5c4e6"} Jul 15 12:28:23.520356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:23.520095 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" event={"ID":"aefe7681-32b1-42c7-9abe-48f40d58eac1","Type":"ContainerStarted","Data":"9a4a35c9a15a8049786c646cb8c498b56fc53dafb3aed0ff223bfce7b027c65f"} Jul 15 12:28:23.520480 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:23.520465 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:28:23.521548 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:23.521527 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mddcc" event={"ID":"ef93d08a-267e-4fdb-b2bb-5c5094c8741f","Type":"ContainerStarted","Data":"9d48249829d38f46ca3647d56b397aacd40d7cb9fed3314cda3bf969e9d9a78a"} Jul 15 12:28:23.567100 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:23.567059 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mddcc" podStartSLOduration=2.289905086 podStartE2EDuration="6.567048208s" totalImagesPullingTime="4.277143122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:23.566857267 +0000 UTC m=+304.117616797" watchObservedRunningTime="2026-07-15 12:28:23.567048208 +0000 UTC m=+304.117807739" Jul 15 12:28:24.528093 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:24.528061 2559 generic.go:356] "Generic (PLEG): container finished" podID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerID="db6d4c8e5f4cff72d394d1e06f5ae4641b6b3cf483a250c9fe0de3e452551283" exitCode=0 Jul 15 12:28:24.528445 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:24.528145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" event={"ID":"aefe7681-32b1-42c7-9abe-48f40d58eac1","Type":"ContainerDied","Data":"db6d4c8e5f4cff72d394d1e06f5ae4641b6b3cf483a250c9fe0de3e452551283"} Jul 15 12:28:25.535016 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:25.534985 2559 generic.go:356] "Generic (PLEG): container finished" podID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerID="e8b4fa03268e9d55ac0b9e25e7074f82ff6c177d58126683e1fcec720d0ff81e" exitCode=0 Jul 15 12:28:25.535405 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:25.535065 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" event={"ID":"aefe7681-32b1-42c7-9abe-48f40d58eac1","Type":"ContainerDied","Data":"e8b4fa03268e9d55ac0b9e25e7074f82ff6c177d58126683e1fcec720d0ff81e"} Jul 15 12:28:26.670969 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.670944 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:26.820159 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.820085 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c5xk\" (UniqueName: \"kubernetes.io/projected/aefe7681-32b1-42c7-9abe-48f40d58eac1-kube-api-access-5c5xk\") pod \"aefe7681-32b1-42c7-9abe-48f40d58eac1\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " Jul 15 12:28:26.820159 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.820144 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-util\") pod \"aefe7681-32b1-42c7-9abe-48f40d58eac1\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " Jul 15 12:28:26.820358 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.820178 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-bundle\") pod \"aefe7681-32b1-42c7-9abe-48f40d58eac1\" (UID: \"aefe7681-32b1-42c7-9abe-48f40d58eac1\") " Jul 15 12:28:26.820878 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.820853 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-bundle" (OuterVolumeSpecName: "bundle") pod "aefe7681-32b1-42c7-9abe-48f40d58eac1" (UID: "aefe7681-32b1-42c7-9abe-48f40d58eac1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:28:26.822132 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.822107 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aefe7681-32b1-42c7-9abe-48f40d58eac1-kube-api-access-5c5xk" (OuterVolumeSpecName: "kube-api-access-5c5xk") pod "aefe7681-32b1-42c7-9abe-48f40d58eac1" (UID: "aefe7681-32b1-42c7-9abe-48f40d58eac1"). InnerVolumeSpecName "kube-api-access-5c5xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:28:26.825731 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.825698 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-util" (OuterVolumeSpecName: "util") pod "aefe7681-32b1-42c7-9abe-48f40d58eac1" (UID: "aefe7681-32b1-42c7-9abe-48f40d58eac1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:28:26.920808 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.920785 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:28:26.920887 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.920823 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c5xk\" (UniqueName: \"kubernetes.io/projected/aefe7681-32b1-42c7-9abe-48f40d58eac1-kube-api-access-5c5xk\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:28:26.920887 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:26.920834 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe7681-32b1-42c7-9abe-48f40d58eac1-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:28:27.545385 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:27.545361 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" Jul 15 12:28:27.545385 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:27.545369 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7db9q" event={"ID":"aefe7681-32b1-42c7-9abe-48f40d58eac1","Type":"ContainerDied","Data":"9a4a35c9a15a8049786c646cb8c498b56fc53dafb3aed0ff223bfce7b027c65f"} Jul 15 12:28:27.545592 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:27.545408 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a4a35c9a15a8049786c646cb8c498b56fc53dafb3aed0ff223bfce7b027c65f" Jul 15 12:28:35.180262 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180228 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk"] Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180532 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="pull" Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180543 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="pull" Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180556 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="util" Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180561 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="util" Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180575 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="extract" Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180582 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="extract" Jul 15 12:28:35.180762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.180636 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aefe7681-32b1-42c7-9abe-48f40d58eac1" containerName="extract" Jul 15 12:28:35.184742 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.184725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.187953 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.187934 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 12:28:35.188883 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.188860 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 12:28:35.188984 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.188905 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vq7wd\"" Jul 15 12:28:35.188984 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.188857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00e13959-81d2-47d0-962f-d4db59be27da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.189094 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.188861 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 12:28:35.189150 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.189090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00e13959-81d2-47d0-962f-d4db59be27da-manifests\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.189150 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.189131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z25t9\" (UniqueName: \"kubernetes.io/projected/00e13959-81d2-47d0-962f-d4db59be27da-kube-api-access-z25t9\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.189261 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.189179 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00e13959-81d2-47d0-962f-d4db59be27da-webhook-cert\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.189261 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.189238 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00e13959-81d2-47d0-962f-d4db59be27da-tmp\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.189881 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.189864 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 12:28:35.203134 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.203113 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk"] Jul 15 12:28:35.289952 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.289920 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00e13959-81d2-47d0-962f-d4db59be27da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.290126 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.289995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00e13959-81d2-47d0-962f-d4db59be27da-manifests\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.290126 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.290031 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z25t9\" (UniqueName: \"kubernetes.io/projected/00e13959-81d2-47d0-962f-d4db59be27da-kube-api-access-z25t9\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.290126 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.290064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00e13959-81d2-47d0-962f-d4db59be27da-webhook-cert\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.290126 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.290109 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00e13959-81d2-47d0-962f-d4db59be27da-tmp\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.290460 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.290435 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00e13959-81d2-47d0-962f-d4db59be27da-manifests\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.292447 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.292421 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00e13959-81d2-47d0-962f-d4db59be27da-tmp\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.292588 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.292573 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00e13959-81d2-47d0-962f-d4db59be27da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.292629 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.292610 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00e13959-81d2-47d0-962f-d4db59be27da-webhook-cert\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.308687 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.308654 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z25t9\" (UniqueName: \"kubernetes.io/projected/00e13959-81d2-47d0-962f-d4db59be27da-kube-api-access-z25t9\") pod \"opendatahub-operator-controller-manager-654f68fd79-6zqrk\" (UID: \"00e13959-81d2-47d0-962f-d4db59be27da\") " pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.495103 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.495071 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:35.660105 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:35.660079 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk"] Jul 15 12:28:35.662216 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:28:35.662190 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e13959_81d2_47d0_962f_d4db59be27da.slice/crio-befff443f4977392cda6872fbaf15dd2050d244bca2e259575f756948168617a WatchSource:0}: Error finding container befff443f4977392cda6872fbaf15dd2050d244bca2e259575f756948168617a: Status 404 returned error can't find the container with id befff443f4977392cda6872fbaf15dd2050d244bca2e259575f756948168617a Jul 15 12:28:36.588168 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:36.588131 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" event={"ID":"00e13959-81d2-47d0-962f-d4db59be27da","Type":"ContainerStarted","Data":"befff443f4977392cda6872fbaf15dd2050d244bca2e259575f756948168617a"} Jul 15 12:28:39.603138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:39.603102 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" event={"ID":"00e13959-81d2-47d0-962f-d4db59be27da","Type":"ContainerStarted","Data":"0befb9734b041a0a6fb91d643dcda979d8c753c10b40dfc78780960ee69f480a"} Jul 15 12:28:40.609072 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:40.609031 2559 generic.go:356] "Generic (PLEG): container finished" podID="00e13959-81d2-47d0-962f-d4db59be27da" containerID="0befb9734b041a0a6fb91d643dcda979d8c753c10b40dfc78780960ee69f480a" exitCode=0 Jul 15 12:28:40.609462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:40.609093 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" event={"ID":"00e13959-81d2-47d0-962f-d4db59be27da","Type":"ContainerDied","Data":"0befb9734b041a0a6fb91d643dcda979d8c753c10b40dfc78780960ee69f480a"} Jul 15 12:28:43.624292 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:43.624202 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" event={"ID":"00e13959-81d2-47d0-962f-d4db59be27da","Type":"ContainerStarted","Data":"017d2402aef58867ca069ff23c3e5dbccffe5b6c1174878e70397018e9c1088c"} Jul 15 12:28:43.624631 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:43.624369 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:43.664418 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:43.664375 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" podStartSLOduration=1.800616565 podStartE2EDuration="8.664361189s" totalImagesPullingTime="5.796943506s" totalInitContainerRuntime="1.066801118s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:43.664300269 +0000 UTC m=+324.215059800" watchObservedRunningTime="2026-07-15 12:28:43.664361189 +0000 UTC m=+324.215120720" Jul 15 12:28:54.632455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:54.632423 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-654f68fd79-6zqrk" Jul 15 12:28:57.651611 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.651573 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9797f867b-j97px"] Jul 15 12:28:57.712774 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.712741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9797f867b-j97px"] Jul 15 12:28:57.712975 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.712887 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.718505 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.718484 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 12:28:57.718934 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.718920 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 12:28:57.797493 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.797458 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwpb6\" (UniqueName: \"kubernetes.io/projected/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-kube-api-access-mwpb6\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.797493 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.797496 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-tls-certs\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.797709 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.797522 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-tmp\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.898622 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.898592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-tmp\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.898794 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.898674 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwpb6\" (UniqueName: \"kubernetes.io/projected/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-kube-api-access-mwpb6\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.898794 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.898696 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-tls-certs\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.900943 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.900922 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-tmp\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.901284 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.901262 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-tls-certs\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:57.933605 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:57.933536 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwpb6\" (UniqueName: \"kubernetes.io/projected/ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3-kube-api-access-mwpb6\") pod \"kube-auth-proxy-9797f867b-j97px\" (UID: \"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3\") " pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:58.022314 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.022283 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" Jul 15 12:28:58.203794 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.203774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9797f867b-j97px"] Jul 15 12:28:58.206455 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:28:58.206432 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7b7ea2_276d_4572_b1b2_2cb4629b5ac3.slice/crio-4de19ceafc9a40c4bc1d701005e36a6f7b3111406b5f5883f4c4d7402bc57c0c WatchSource:0}: Error finding container 4de19ceafc9a40c4bc1d701005e36a6f7b3111406b5f5883f4c4d7402bc57c0c: Status 404 returned error can't find the container with id 4de19ceafc9a40c4bc1d701005e36a6f7b3111406b5f5883f4c4d7402bc57c0c Jul 15 12:28:58.319559 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.319532 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cwkbf"] Jul 15 12:28:58.322859 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.322843 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:58.341676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.341653 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 12:28:58.341955 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.341942 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6nckc\"" Jul 15 12:28:58.382036 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.382015 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cwkbf"] Jul 15 12:28:58.401688 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.401668 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ll7\" (UniqueName: \"kubernetes.io/projected/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-kube-api-access-c8ll7\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:58.401783 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.401706 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:58.502787 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.502755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ll7\" (UniqueName: \"kubernetes.io/projected/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-kube-api-access-c8ll7\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:58.502927 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.502832 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:58.502988 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:28:58.502969 2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 12:28:58.503047 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:28:58.503037 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert podName:ecb74447-252c-4ce1-978b-c1cba7ddc4fa nodeName:}" failed. No retries permitted until 2026-07-15 12:28:59.003020794 +0000 UTC m=+339.553780308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert") pod "odh-model-controller-7998c67dc4-cwkbf" (UID: "ecb74447-252c-4ce1-978b-c1cba7ddc4fa") : secret "odh-model-controller-webhook-cert" not found Jul 15 12:28:58.631064 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.631044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ll7\" (UniqueName: \"kubernetes.io/projected/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-kube-api-access-c8ll7\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:58.698104 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:58.698070 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" event={"ID":"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3","Type":"ContainerStarted","Data":"4de19ceafc9a40c4bc1d701005e36a6f7b3111406b5f5883f4c4d7402bc57c0c"} Jul 15 12:28:59.007608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:28:59.007575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:28:59.007784 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:28:59.007744 2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 12:28:59.007879 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:28:59.007839 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert podName:ecb74447-252c-4ce1-978b-c1cba7ddc4fa nodeName:}" failed. No retries permitted until 2026-07-15 12:29:00.007818049 +0000 UTC m=+340.558577572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert") pod "odh-model-controller-7998c67dc4-cwkbf" (UID: "ecb74447-252c-4ce1-978b-c1cba7ddc4fa") : secret "odh-model-controller-webhook-cert" not found Jul 15 12:29:00.018010 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.017972 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:29:00.020823 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.020782 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ecb74447-252c-4ce1-978b-c1cba7ddc4fa-cert\") pod \"odh-model-controller-7998c67dc4-cwkbf\" (UID: \"ecb74447-252c-4ce1-978b-c1cba7ddc4fa\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:29:00.133098 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.133065 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:29:00.304886 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.304857 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cwkbf"] Jul 15 12:29:00.306825 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:00.306778 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb74447_252c_4ce1_978b_c1cba7ddc4fa.slice/crio-dff3ef3d1b8c9f6c4b058f8d3b6f6f6678f3ea1b1650b5122d317a2ad473b1b0 WatchSource:0}: Error finding container dff3ef3d1b8c9f6c4b058f8d3b6f6f6678f3ea1b1650b5122d317a2ad473b1b0: Status 404 returned error can't find the container with id dff3ef3d1b8c9f6c4b058f8d3b6f6f6678f3ea1b1650b5122d317a2ad473b1b0 Jul 15 12:29:00.334107 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.334082 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vff75"] Jul 15 12:29:00.337141 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.337120 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:00.340334 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.340312 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nz96j\"" Jul 15 12:29:00.340501 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.340485 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 12:29:00.352414 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.352338 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vff75"] Jul 15 12:29:00.423032 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.422993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:00.423208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.423122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2ndf\" (UniqueName: \"kubernetes.io/projected/e6cdba19-5aaf-4219-a329-1e733a2391db-kube-api-access-v2ndf\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:00.524404 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.524368 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:00.524570 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.524483 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2ndf\" (UniqueName: \"kubernetes.io/projected/e6cdba19-5aaf-4219-a329-1e733a2391db-kube-api-access-v2ndf\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:00.524773 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:29:00.524749 2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 12:29:00.524915 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:29:00.524860 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs podName:e6cdba19-5aaf-4219-a329-1e733a2391db nodeName:}" failed. No retries permitted until 2026-07-15 12:29:01.024837113 +0000 UTC m=+341.575596647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs") pod "model-serving-api-54d89bd5fc-vff75" (UID: "e6cdba19-5aaf-4219-a329-1e733a2391db") : secret "model-serving-api-tls" not found Jul 15 12:29:00.538952 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.538925 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2ndf\" (UniqueName: \"kubernetes.io/projected/e6cdba19-5aaf-4219-a329-1e733a2391db-kube-api-access-v2ndf\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:00.581235 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.581174 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk"] Jul 15 12:29:00.585336 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.585316 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.590926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.590899 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 12:29:00.591624 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.591474 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 12:29:00.591736 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.591676 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 12:29:00.591926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.591875 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 12:29:00.592230 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.592210 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-68vdk\"" Jul 15 12:29:00.612678 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.612638 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk"] Jul 15 12:29:00.710699 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.710666 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" event={"ID":"ecb74447-252c-4ce1-978b-c1cba7ddc4fa","Type":"ContainerStarted","Data":"dff3ef3d1b8c9f6c4b058f8d3b6f6f6678f3ea1b1650b5122d317a2ad473b1b0"} Jul 15 12:29:00.727167 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjrkc\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-kube-api-access-sjrkc\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.727315 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/704d71c6-b6cf-415c-8a0d-e17459708809-local-certs\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.727315 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727269 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.727315 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.727455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-cacerts\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.727455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.727541 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.727466 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-istio-token\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828212 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/704d71c6-b6cf-415c-8a0d-e17459708809-local-certs\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828400 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828400 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828262 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828400 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828296 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-cacerts\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828400 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828400 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828398 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-istio-token\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.828654 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.828446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjrkc\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-kube-api-access-sjrkc\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.829545 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.829494 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.831058 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.831031 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/704d71c6-b6cf-415c-8a0d-e17459708809-local-certs\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.831390 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.831334 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-cacerts\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.831483 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.831412 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.831551 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.831510 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.847880 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.847857 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-istio-token\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.848185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.848168 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjrkc\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-kube-api-access-sjrkc\") pod \"istiod-openshift-gateway-7574459656-f6ltk\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:00.897258 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:00.897228 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:01.030391 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:01.030303 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:01.030855 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:29:01.030472 2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 12:29:01.030855 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:29:01.030579 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs podName:e6cdba19-5aaf-4219-a329-1e733a2391db nodeName:}" failed. No retries permitted until 2026-07-15 12:29:02.030556534 +0000 UTC m=+342.581316064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs") pod "model-serving-api-54d89bd5fc-vff75" (UID: "e6cdba19-5aaf-4219-a329-1e733a2391db") : secret "model-serving-api-tls" not found Jul 15 12:29:01.171869 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:01.171844 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk"] Jul 15 12:29:01.174363 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:01.174330 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704d71c6_b6cf_415c_8a0d_e17459708809.slice/crio-2f45712d61a8337f1d8454a18dba7ba260ac7f12c82aba1fbb6837d706887cf1 WatchSource:0}: Error finding container 2f45712d61a8337f1d8454a18dba7ba260ac7f12c82aba1fbb6837d706887cf1: Status 404 returned error can't find the container with id 2f45712d61a8337f1d8454a18dba7ba260ac7f12c82aba1fbb6837d706887cf1 Jul 15 12:29:01.718355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:01.718301 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" event={"ID":"ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3","Type":"ContainerStarted","Data":"360bfa7df81c4174fbdb37e66012b52425ac16f11bbf1a4cd0004cdbb4fbb678"} Jul 15 12:29:01.719688 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:01.719660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" event={"ID":"704d71c6-b6cf-415c-8a0d-e17459708809","Type":"ContainerStarted","Data":"2f45712d61a8337f1d8454a18dba7ba260ac7f12c82aba1fbb6837d706887cf1"} Jul 15 12:29:01.747730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:01.747676 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9797f867b-j97px" podStartSLOduration=1.880081085 podStartE2EDuration="4.747661268s" totalImagesPullingTime="2.867580183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:01.747321856 +0000 UTC m=+342.298081391" watchObservedRunningTime="2026-07-15 12:29:01.747661268 +0000 UTC m=+342.298420799" Jul 15 12:29:02.041604 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:02.041523 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:02.043758 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:02.043739 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6cdba19-5aaf-4219-a329-1e733a2391db-tls-certs\") pod \"model-serving-api-54d89bd5fc-vff75\" (UID: \"e6cdba19-5aaf-4219-a329-1e733a2391db\") " pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:02.148951 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:02.148917 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:02.306133 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:02.306103 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vff75"] Jul 15 12:29:02.309478 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:02.309450 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cdba19_5aaf_4219_a329_1e733a2391db.slice/crio-33a670fcd7fab00c90197dde04f05bbc80e01d3a58fc3f64d732f5e3fd26da49 WatchSource:0}: Error finding container 33a670fcd7fab00c90197dde04f05bbc80e01d3a58fc3f64d732f5e3fd26da49: Status 404 returned error can't find the container with id 33a670fcd7fab00c90197dde04f05bbc80e01d3a58fc3f64d732f5e3fd26da49 Jul 15 12:29:02.726793 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:02.726760 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" event={"ID":"e6cdba19-5aaf-4219-a329-1e733a2391db","Type":"ContainerStarted","Data":"33a670fcd7fab00c90197dde04f05bbc80e01d3a58fc3f64d732f5e3fd26da49"} Jul 15 12:29:04.746387 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:04.746349 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" event={"ID":"ecb74447-252c-4ce1-978b-c1cba7ddc4fa","Type":"ContainerStarted","Data":"15f32a6ec080211e857bb1dbac9084c4a188ae4cbe362b5d4d74fe3cf483e70d"} Jul 15 12:29:04.746902 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:04.746446 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:29:04.782723 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:04.782662 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" podStartSLOduration=3.120834597 podStartE2EDuration="6.782640346s" totalImagesPullingTime="3.661805749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:04.778932656 +0000 UTC m=+345.329692211" watchObservedRunningTime="2026-07-15 12:29:04.782640346 +0000 UTC m=+345.333399879" Jul 15 12:29:06.271485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.271455 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:29:06.271756 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.271534 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:29:06.759065 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.759024 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" event={"ID":"e6cdba19-5aaf-4219-a329-1e733a2391db","Type":"ContainerStarted","Data":"53ed07b8b38cc45a126c5768547e1174f2cffa4e8f3f59613d26fd1288574f5e"} Jul 15 12:29:06.759328 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.759304 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:06.760715 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.760690 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" event={"ID":"704d71c6-b6cf-415c-8a0d-e17459708809","Type":"ContainerStarted","Data":"d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044"} Jul 15 12:29:06.760867 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.760848 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:06.808591 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.808539 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" podStartSLOduration=2.8551636460000003 podStartE2EDuration="6.808524763s" totalImagesPullingTime="3.953361117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:06.800666363 +0000 UTC m=+347.351425908" watchObservedRunningTime="2026-07-15 12:29:06.808524763 +0000 UTC m=+347.359284295" Jul 15 12:29:06.859282 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:06.859222 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" podStartSLOduration=1.764646119 podStartE2EDuration="6.859205455s" totalImagesPullingTime="5.094559336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:06.859187334 +0000 UTC m=+347.409946890" watchObservedRunningTime="2026-07-15 12:29:06.859205455 +0000 UTC m=+347.409964987" Jul 15 12:29:07.768062 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:07.768036 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:07.944491 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:07.944465 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m"] Jul 15 12:29:07.950250 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:07.950226 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:07.954364 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:07.954337 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fs852\"" Jul 15 12:29:08.076573 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.076490 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m"] Jul 15 12:29:08.103355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103320 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103567 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103567 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103408 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103567 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103433 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbck6\" (UniqueName: \"kubernetes.io/projected/40c38105-86fe-4c9b-9c8c-8958a54f7203-kube-api-access-fbck6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103687 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103687 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103616 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c38105-86fe-4c9b-9c8c-8958a54f7203-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103687 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.103815 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.103729 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205220 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205185 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205226 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205264 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205316 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205343 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbck6\" (UniqueName: \"kubernetes.io/projected/40c38105-86fe-4c9b-9c8c-8958a54f7203-kube-api-access-fbck6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205394 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c38105-86fe-4c9b-9c8c-8958a54f7203-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205456 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205495 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.205693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.205674 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.206047 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.206025 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.206144 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.206124 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.206248 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.206230 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.206293 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.206279 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c38105-86fe-4c9b-9c8c-8958a54f7203-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.207694 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.207675 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.208113 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.208090 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.235820 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.235785 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c38105-86fe-4c9b-9c8c-8958a54f7203-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.235953 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.235932 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbck6\" (UniqueName: \"kubernetes.io/projected/40c38105-86fe-4c9b-9c8c-8958a54f7203-kube-api-access-fbck6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fqz8m\" (UID: \"40c38105-86fe-4c9b-9c8c-8958a54f7203\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.261310 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.261281 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:08.521186 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.521159 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m"] Jul 15 12:29:08.522912 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:08.522883 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c38105_86fe_4c9b_9c8c_8958a54f7203.slice/crio-1283654bb48e8afed03a25b194fd806b56141e298dd61968c6a8481e886396b3 WatchSource:0}: Error finding container 1283654bb48e8afed03a25b194fd806b56141e298dd61968c6a8481e886396b3: Status 404 returned error can't find the container with id 1283654bb48e8afed03a25b194fd806b56141e298dd61968c6a8481e886396b3 Jul 15 12:29:08.772362 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:08.772327 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" event={"ID":"40c38105-86fe-4c9b-9c8c-8958a54f7203","Type":"ContainerStarted","Data":"1283654bb48e8afed03a25b194fd806b56141e298dd61968c6a8481e886396b3"} Jul 15 12:29:10.524116 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:10.524077 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk"] Jul 15 12:29:10.524594 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:10.524320 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" podUID="704d71c6-b6cf-415c-8a0d-e17459708809" containerName="discovery" containerID="cri-o://d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044" gracePeriod=30 Jul 15 12:29:10.767407 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:10.767368 2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-f6ltk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:8080/ready\": dial tcp 10.132.0.36:8080: connect: connection refused" start-of-body= Jul 15 12:29:10.767570 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:10.767434 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" podUID="704d71c6-b6cf-415c-8a0d-e17459708809" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.36:8080/ready\": dial tcp 10.132.0.36:8080: connect: connection refused" Jul 15 12:29:11.042472 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.042428 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:29:11.042592 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.042530 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:29:11.042592 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.042581 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:29:11.129414 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.129387 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:11.234530 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.234449 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-istio-token\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.234530 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.234499 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-cacerts\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.234720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.234546 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-ca-configmap\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.234720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.234571 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-dns-cert\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.234720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.234609 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/704d71c6-b6cf-415c-8a0d-e17459708809-local-certs\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.235349 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.235053 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:29:11.235349 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.235137 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjrkc\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-kube-api-access-sjrkc\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.235349 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.235225 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-kubeconfig\") pod \"704d71c6-b6cf-415c-8a0d-e17459708809\" (UID: \"704d71c6-b6cf-415c-8a0d-e17459708809\") " Jul 15 12:29:11.235568 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.235523 2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-ca-configmap\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.237457 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.237428 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704d71c6-b6cf-415c-8a0d-e17459708809-local-certs" (OuterVolumeSpecName: "local-certs") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:29:11.237757 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.237703 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:29:11.237757 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.237720 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-kube-api-access-sjrkc" (OuterVolumeSpecName: "kube-api-access-sjrkc") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "kube-api-access-sjrkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:29:11.238014 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.237971 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-cacerts" (OuterVolumeSpecName: "cacerts") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:29:11.238125 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.238097 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-istio-token" (OuterVolumeSpecName: "istio-token") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:29:11.238458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.238431 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "704d71c6-b6cf-415c-8a0d-e17459708809" (UID: "704d71c6-b6cf-415c-8a0d-e17459708809"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:29:11.336647 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.336614 2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/704d71c6-b6cf-415c-8a0d-e17459708809-local-certs\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.336647 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.336642 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjrkc\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-kube-api-access-sjrkc\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.336647 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.336651 2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-kubeconfig\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.336647 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.336659 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704d71c6-b6cf-415c-8a0d-e17459708809-istio-token\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.336988 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.336668 2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-cacerts\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.336988 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.336675 2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/704d71c6-b6cf-415c-8a0d-e17459708809-istio-csr-dns-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:11.789931 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.789900 2559 generic.go:356] "Generic (PLEG): container finished" podID="704d71c6-b6cf-415c-8a0d-e17459708809" containerID="d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044" exitCode=0 Jul 15 12:29:11.790392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.789945 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" Jul 15 12:29:11.790392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.789979 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" event={"ID":"704d71c6-b6cf-415c-8a0d-e17459708809","Type":"ContainerDied","Data":"d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044"} Jul 15 12:29:11.790392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.790042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk" event={"ID":"704d71c6-b6cf-415c-8a0d-e17459708809","Type":"ContainerDied","Data":"2f45712d61a8337f1d8454a18dba7ba260ac7f12c82aba1fbb6837d706887cf1"} Jul 15 12:29:11.790392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.790065 2559 scope.go:117] "RemoveContainer" containerID="d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044" Jul 15 12:29:11.791731 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.791708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" event={"ID":"40c38105-86fe-4c9b-9c8c-8958a54f7203","Type":"ContainerStarted","Data":"5e3813462dc114990f3b1f43fee6f5140653f2ed2c007c65bcee2128a2fa7ed5"} Jul 15 12:29:11.799070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.799053 2559 scope.go:117] "RemoveContainer" containerID="d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044" Jul 15 12:29:11.799334 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:29:11.799317 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044\": container with ID starting with d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044 not found: ID does not exist" containerID="d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044" Jul 15 12:29:11.799396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.799343 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044"} err="failed to get container status \"d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044\": rpc error: code = NotFound desc = could not find container \"d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044\": container with ID starting with d616e4f982c717a1369371d3e2c78b1179901d0e8387189bd16641ed406c4044 not found: ID does not exist" Jul 15 12:29:11.945866 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.945788 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" podStartSLOduration=2.428854073 podStartE2EDuration="4.945767334s" totalImagesPullingTime="2.516913261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:11.942984364 +0000 UTC m=+352.493743924" watchObservedRunningTime="2026-07-15 12:29:11.945767334 +0000 UTC m=+352.496526867" Jul 15 12:29:11.992341 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:11.992311 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk"] Jul 15 12:29:12.004822 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.004777 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f6ltk"] Jul 15 12:29:12.110695 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.110613 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="704d71c6-b6cf-415c-8a0d-e17459708809" path="/var/lib/kubelet/pods/704d71c6-b6cf-415c-8a0d-e17459708809/volumes" Jul 15 12:29:12.123154 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.123116 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4"] Jul 15 12:29:12.123930 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.123900 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704d71c6-b6cf-415c-8a0d-e17459708809" containerName="discovery" Jul 15 12:29:12.123930 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.123931 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="704d71c6-b6cf-415c-8a0d-e17459708809" containerName="discovery" Jul 15 12:29:12.124376 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.124354 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="704d71c6-b6cf-415c-8a0d-e17459708809" containerName="discovery" Jul 15 12:29:12.128872 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.128847 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck"] Jul 15 12:29:12.129003 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.128953 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.132494 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.132470 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-lchlz\"" Jul 15 12:29:12.132680 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.132659 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.133411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.133390 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 12:29:12.140356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.140201 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 12:29:12.141826 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.140689 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5b9nc\"" Jul 15 12:29:12.142387 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.142367 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4"] Jul 15 12:29:12.156469 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.156442 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck"] Jul 15 12:29:12.248485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.248407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99lsf\" (UniqueName: \"kubernetes.io/projected/f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466-kube-api-access-99lsf\") pod \"llmisvc-controller-manager-777c6bd7d5-xm9ck\" (UID: \"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.248485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.248446 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8858c84-c69a-4b01-bdbc-38b065a7508b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tkrr4\" (UID: \"d8858c84-c69a-4b01-bdbc-38b065a7508b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.248715 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.248550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xm9ck\" (UID: \"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.248715 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.248594 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdccw\" (UniqueName: \"kubernetes.io/projected/d8858c84-c69a-4b01-bdbc-38b065a7508b-kube-api-access-hdccw\") pod \"kserve-controller-manager-d6f4b65b8-tkrr4\" (UID: \"d8858c84-c69a-4b01-bdbc-38b065a7508b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.262374 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.262350 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:12.266980 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.266957 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:12.350160 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.350126 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99lsf\" (UniqueName: \"kubernetes.io/projected/f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466-kube-api-access-99lsf\") pod \"llmisvc-controller-manager-777c6bd7d5-xm9ck\" (UID: \"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.350343 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.350167 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8858c84-c69a-4b01-bdbc-38b065a7508b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tkrr4\" (UID: \"d8858c84-c69a-4b01-bdbc-38b065a7508b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.350343 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.350215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xm9ck\" (UID: \"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.350343 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.350235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdccw\" (UniqueName: \"kubernetes.io/projected/d8858c84-c69a-4b01-bdbc-38b065a7508b-kube-api-access-hdccw\") pod \"kserve-controller-manager-d6f4b65b8-tkrr4\" (UID: \"d8858c84-c69a-4b01-bdbc-38b065a7508b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.352632 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.352605 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8858c84-c69a-4b01-bdbc-38b065a7508b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tkrr4\" (UID: \"d8858c84-c69a-4b01-bdbc-38b065a7508b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.352783 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.352761 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xm9ck\" (UID: \"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.371115 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.371047 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99lsf\" (UniqueName: \"kubernetes.io/projected/f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466-kube-api-access-99lsf\") pod \"llmisvc-controller-manager-777c6bd7d5-xm9ck\" (UID: \"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.371563 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.371540 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdccw\" (UniqueName: \"kubernetes.io/projected/d8858c84-c69a-4b01-bdbc-38b065a7508b-kube-api-access-hdccw\") pod \"kserve-controller-manager-d6f4b65b8-tkrr4\" (UID: \"d8858c84-c69a-4b01-bdbc-38b065a7508b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.443285 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.443250 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:12.450768 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.450742 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:12.608515 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:12.608480 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8858c84_c69a_4b01_bdbc_38b065a7508b.slice/crio-f3c9d78cf4fe681a4fa77d8f67725b14417b32bbbb6688d8ac19755c178eb92f WatchSource:0}: Error finding container f3c9d78cf4fe681a4fa77d8f67725b14417b32bbbb6688d8ac19755c178eb92f: Status 404 returned error can't find the container with id f3c9d78cf4fe681a4fa77d8f67725b14417b32bbbb6688d8ac19755c178eb92f Jul 15 12:29:12.612296 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.612276 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4"] Jul 15 12:29:12.659044 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.659020 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck"] Jul 15 12:29:12.661426 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:12.661394 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf9d8e1c7_3a6e_4dc6_ba71_eea13b56b466.slice/crio-0542e511c376921b42c8a793864e3b7445c40ea30e782edf03ced1d23c5f4771 WatchSource:0}: Error finding container 0542e511c376921b42c8a793864e3b7445c40ea30e782edf03ced1d23c5f4771: Status 404 returned error can't find the container with id 0542e511c376921b42c8a793864e3b7445c40ea30e782edf03ced1d23c5f4771 Jul 15 12:29:12.797821 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.797770 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" event={"ID":"d8858c84-c69a-4b01-bdbc-38b065a7508b","Type":"ContainerStarted","Data":"f3c9d78cf4fe681a4fa77d8f67725b14417b32bbbb6688d8ac19755c178eb92f"} Jul 15 12:29:12.799884 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.799860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" event={"ID":"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466","Type":"ContainerStarted","Data":"0542e511c376921b42c8a793864e3b7445c40ea30e782edf03ced1d23c5f4771"} Jul 15 12:29:12.800204 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.800189 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:12.801174 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:12.801155 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fqz8m" Jul 15 12:29:15.754948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:15.754918 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cwkbf" Jul 15 12:29:15.829416 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:15.828786 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" event={"ID":"d8858c84-c69a-4b01-bdbc-38b065a7508b","Type":"ContainerStarted","Data":"1a87a85e1ce501a3142d83d8352a260651d93352e189d0a573f037bfedef850c"} Jul 15 12:29:15.829416 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:15.828880 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:15.869173 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:15.869124 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" podStartSLOduration=1.6434139349999999 podStartE2EDuration="3.86910728s" totalImagesPullingTime="2.225693345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:15.865899579 +0000 UTC m=+356.416659134" watchObservedRunningTime="2026-07-15 12:29:15.86910728 +0000 UTC m=+356.419866840" Jul 15 12:29:16.835340 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:16.835257 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" event={"ID":"f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466","Type":"ContainerStarted","Data":"6fff6a76ff3ed822007e81b1e923cc207133128630e8a19d34ae070f028f5bbb"} Jul 15 12:29:16.835691 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:16.835487 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:16.886600 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:16.886546 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" podStartSLOduration=1.010206194 podStartE2EDuration="4.886527518s" totalImagesPullingTime="3.876321324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:16.884762594 +0000 UTC m=+357.435522160" watchObservedRunningTime="2026-07-15 12:29:16.886527518 +0000 UTC m=+357.437287051" Jul 15 12:29:17.770294 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:17.770260 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vff75" Jul 15 12:29:46.840770 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:46.840731 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tkrr4" Jul 15 12:29:47.844153 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:47.844118 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xm9ck" Jul 15 12:29:58.487608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.487571 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hvhkz"] Jul 15 12:29:58.498768 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.498742 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:29:58.505298 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.505264 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 12:29:58.505668 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.505647 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 12:29:58.513507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.513487 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zzzzz\"" Jul 15 12:29:58.518304 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.518281 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hvhkz"] Jul 15 12:29:58.654456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.654415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvsf5\" (UniqueName: \"kubernetes.io/projected/49f6b97f-15f5-4004-a628-942edbe13992-kube-api-access-wvsf5\") pod \"kuadrant-operator-catalog-hvhkz\" (UID: \"49f6b97f-15f5-4004-a628-942edbe13992\") " pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:29:58.755162 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.755067 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hvhkz"] Jul 15 12:29:58.755162 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.755138 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvsf5\" (UniqueName: \"kubernetes.io/projected/49f6b97f-15f5-4004-a628-942edbe13992-kube-api-access-wvsf5\") pod \"kuadrant-operator-catalog-hvhkz\" (UID: \"49f6b97f-15f5-4004-a628-942edbe13992\") " pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:29:58.755356 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:29:58.755309 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wvsf5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" podUID="49f6b97f-15f5-4004-a628-942edbe13992" Jul 15 12:29:58.769780 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.769753 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvsf5\" (UniqueName: \"kubernetes.io/projected/49f6b97f-15f5-4004-a628-942edbe13992-kube-api-access-wvsf5\") pod \"kuadrant-operator-catalog-hvhkz\" (UID: \"49f6b97f-15f5-4004-a628-942edbe13992\") " pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:29:58.968995 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.968960 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dmhq5"] Jul 15 12:29:58.972557 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.972539 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:29:58.987554 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:58.987530 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dmhq5"] Jul 15 12:29:59.039607 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.039520 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:29:59.046455 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.046434 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:29:59.159033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.158998 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvsf5\" (UniqueName: \"kubernetes.io/projected/49f6b97f-15f5-4004-a628-942edbe13992-kube-api-access-wvsf5\") pod \"49f6b97f-15f5-4004-a628-942edbe13992\" (UID: \"49f6b97f-15f5-4004-a628-942edbe13992\") " Jul 15 12:29:59.159206 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.159183 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d979f\" (UniqueName: \"kubernetes.io/projected/9ec0e4a2-d10d-4e20-adbb-75f805848c47-kube-api-access-d979f\") pod \"kuadrant-operator-catalog-dmhq5\" (UID: \"9ec0e4a2-d10d-4e20-adbb-75f805848c47\") " pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:29:59.161172 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.161144 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49f6b97f-15f5-4004-a628-942edbe13992-kube-api-access-wvsf5" (OuterVolumeSpecName: "kube-api-access-wvsf5") pod "49f6b97f-15f5-4004-a628-942edbe13992" (UID: "49f6b97f-15f5-4004-a628-942edbe13992"). InnerVolumeSpecName "kube-api-access-wvsf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:29:59.260175 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.260140 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d979f\" (UniqueName: \"kubernetes.io/projected/9ec0e4a2-d10d-4e20-adbb-75f805848c47-kube-api-access-d979f\") pod \"kuadrant-operator-catalog-dmhq5\" (UID: \"9ec0e4a2-d10d-4e20-adbb-75f805848c47\") " pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:29:59.260361 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.260267 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvsf5\" (UniqueName: \"kubernetes.io/projected/49f6b97f-15f5-4004-a628-942edbe13992-kube-api-access-wvsf5\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:29:59.270537 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.270505 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d979f\" (UniqueName: \"kubernetes.io/projected/9ec0e4a2-d10d-4e20-adbb-75f805848c47-kube-api-access-d979f\") pod \"kuadrant-operator-catalog-dmhq5\" (UID: \"9ec0e4a2-d10d-4e20-adbb-75f805848c47\") " pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:29:59.282552 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.282530 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:29:59.419866 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:29:59.419842 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dmhq5"] Jul 15 12:29:59.421081 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:29:59.421053 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec0e4a2_d10d_4e20_adbb_75f805848c47.slice/crio-fbd225a69beb0d1ddae62db0d644de6cb6568a4f956ed066908ae6bc3566de0c WatchSource:0}: Error finding container fbd225a69beb0d1ddae62db0d644de6cb6568a4f956ed066908ae6bc3566de0c: Status 404 returned error can't find the container with id fbd225a69beb0d1ddae62db0d644de6cb6568a4f956ed066908ae6bc3566de0c Jul 15 12:30:00.051097 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:00.051064 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hvhkz" Jul 15 12:30:00.051728 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:00.051250 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" event={"ID":"9ec0e4a2-d10d-4e20-adbb-75f805848c47","Type":"ContainerStarted","Data":"fbd225a69beb0d1ddae62db0d644de6cb6568a4f956ed066908ae6bc3566de0c"} Jul 15 12:30:00.090356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:00.090322 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hvhkz"] Jul 15 12:30:00.094563 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:00.094534 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hvhkz"] Jul 15 12:30:00.114089 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:00.114050 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49f6b97f-15f5-4004-a628-942edbe13992" path="/var/lib/kubelet/pods/49f6b97f-15f5-4004-a628-942edbe13992/volumes" Jul 15 12:30:03.067022 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:03.066990 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" event={"ID":"9ec0e4a2-d10d-4e20-adbb-75f805848c47","Type":"ContainerStarted","Data":"ac0e2e43e5f8150c24664799244f933d4bc560bfa19d8d8ce97abf322b4a55a2"} Jul 15 12:30:09.282716 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:09.282678 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:30:09.283252 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:09.282761 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:30:09.305785 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:09.305756 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:30:09.335115 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:09.335067 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" podStartSLOduration=7.888715209 podStartE2EDuration="11.335052116s" totalImagesPullingTime="3.446336907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:03.100664492 +0000 UTC m=+403.651424024" watchObservedRunningTime="2026-07-15 12:30:09.335052116 +0000 UTC m=+409.885811648" Jul 15 12:30:10.123950 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:10.123924 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dmhq5" Jul 15 12:30:14.209220 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.209184 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6"] Jul 15 12:30:14.212988 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.212965 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.216042 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.216023 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7c6ng\"" Jul 15 12:30:14.222664 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.222643 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6"] Jul 15 12:30:14.274698 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.274663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt5ds\" (UniqueName: \"kubernetes.io/projected/5596d224-0d77-437a-ad3d-971e8c95b127-kube-api-access-dt5ds\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.274698 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.274696 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.274924 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.274723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.375739 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.375702 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5ds\" (UniqueName: \"kubernetes.io/projected/5596d224-0d77-437a-ad3d-971e8c95b127-kube-api-access-dt5ds\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.375739 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.375741 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.375983 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.375874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.376146 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.376130 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.376187 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.376172 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.387693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.387669 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt5ds\" (UniqueName: \"kubernetes.io/projected/5596d224-0d77-437a-ad3d-971e8c95b127-kube-api-access-dt5ds\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.522759 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.522673 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:14.633587 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.633558 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76"] Jul 15 12:30:14.637716 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.637696 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.654656 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.654635 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76"] Jul 15 12:30:14.671686 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.671658 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6"] Jul 15 12:30:14.674200 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:14.674169 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5596d224_0d77_437a_ad3d_971e8c95b127.slice/crio-1e975ccd2be734b4745531fda9feefee89d4f52bb17cb802571dd372004a68bd WatchSource:0}: Error finding container 1e975ccd2be734b4745531fda9feefee89d4f52bb17cb802571dd372004a68bd: Status 404 returned error can't find the container with id 1e975ccd2be734b4745531fda9feefee89d4f52bb17cb802571dd372004a68bd Jul 15 12:30:14.678613 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.678593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gkcz\" (UniqueName: \"kubernetes.io/projected/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-kube-api-access-2gkcz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.678699 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.678640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.678699 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.678683 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.779954 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.779924 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gkcz\" (UniqueName: \"kubernetes.io/projected/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-kube-api-access-2gkcz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.780068 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.779979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.780068 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.780002 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.780374 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.780357 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.780374 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.780369 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.789578 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.789552 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gkcz\" (UniqueName: \"kubernetes.io/projected/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-kube-api-access-2gkcz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:14.948090 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:14.948059 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:15.061091 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.061034 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg"] Jul 15 12:30:15.065785 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.065765 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.081975 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.081940 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74bn4\" (UniqueName: \"kubernetes.io/projected/914c99bc-2302-4dc1-bf11-028d24766371-kube-api-access-74bn4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.081975 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.081972 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.082157 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.082101 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.084185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.084164 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg"] Jul 15 12:30:15.113328 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.112385 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76"] Jul 15 12:30:15.128998 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.128970 2559 generic.go:356] "Generic (PLEG): container finished" podID="5596d224-0d77-437a-ad3d-971e8c95b127" containerID="f6a2934abffa1b2dd6b726e3e2c0d6fd56690f2fb2e584569e8e34d59b1c59c9" exitCode=0 Jul 15 12:30:15.129110 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.129038 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" event={"ID":"5596d224-0d77-437a-ad3d-971e8c95b127","Type":"ContainerDied","Data":"f6a2934abffa1b2dd6b726e3e2c0d6fd56690f2fb2e584569e8e34d59b1c59c9"} Jul 15 12:30:15.129110 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.129095 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" event={"ID":"5596d224-0d77-437a-ad3d-971e8c95b127","Type":"ContainerStarted","Data":"1e975ccd2be734b4745531fda9feefee89d4f52bb17cb802571dd372004a68bd"} Jul 15 12:30:15.130441 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.130414 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" event={"ID":"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f","Type":"ContainerStarted","Data":"88a9b373cc313fce027328551dd73a8887f417891038f694d9f20ff01d294d79"} Jul 15 12:30:15.182943 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.182918 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74bn4\" (UniqueName: \"kubernetes.io/projected/914c99bc-2302-4dc1-bf11-028d24766371-kube-api-access-74bn4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.183060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.182951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.183060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.183001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.183363 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.183342 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.183401 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.183364 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.193687 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.193659 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74bn4\" (UniqueName: \"kubernetes.io/projected/914c99bc-2302-4dc1-bf11-028d24766371-kube-api-access-74bn4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.376033 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.375935 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:15.434187 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.434161 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2"] Jul 15 12:30:15.438611 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.438592 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.450409 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.450385 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2"] Jul 15 12:30:15.486050 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.486021 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.486180 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.486147 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.486231 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.486200 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv89r\" (UniqueName: \"kubernetes.io/projected/23c61466-535d-433d-ba92-d9eaf1684a9e-kube-api-access-zv89r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.518558 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.518534 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg"] Jul 15 12:30:15.519725 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:15.519695 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914c99bc_2302_4dc1_bf11_028d24766371.slice/crio-0f7d16feb0b9efb03502ad0cfc9a8e0cb384d712e2c6957657fd1f8f3dbba2a5 WatchSource:0}: Error finding container 0f7d16feb0b9efb03502ad0cfc9a8e0cb384d712e2c6957657fd1f8f3dbba2a5: Status 404 returned error can't find the container with id 0f7d16feb0b9efb03502ad0cfc9a8e0cb384d712e2c6957657fd1f8f3dbba2a5 Jul 15 12:30:15.587123 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.587094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.587232 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.587161 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv89r\" (UniqueName: \"kubernetes.io/projected/23c61466-535d-433d-ba92-d9eaf1684a9e-kube-api-access-zv89r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.587232 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.587223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.587543 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.587525 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.587580 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.587542 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.597311 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.597288 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv89r\" (UniqueName: \"kubernetes.io/projected/23c61466-535d-433d-ba92-d9eaf1684a9e-kube-api-access-zv89r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.748471 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.748435 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:15.894199 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:15.894177 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2"] Jul 15 12:30:15.900976 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:15.900939 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c61466_535d_433d_ba92_d9eaf1684a9e.slice/crio-94f388b28c09eab79c4aa6b88f13e298c0a9c898cb4bfc45b8a25bae70997324 WatchSource:0}: Error finding container 94f388b28c09eab79c4aa6b88f13e298c0a9c898cb4bfc45b8a25bae70997324: Status 404 returned error can't find the container with id 94f388b28c09eab79c4aa6b88f13e298c0a9c898cb4bfc45b8a25bae70997324 Jul 15 12:30:16.137129 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.137041 2559 generic.go:356] "Generic (PLEG): container finished" podID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerID="23e406d79201b78107d77b1003e2a23586a565d946bf203490031f97658dbd87" exitCode=0 Jul 15 12:30:16.137291 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.137126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" event={"ID":"23c61466-535d-433d-ba92-d9eaf1684a9e","Type":"ContainerDied","Data":"23e406d79201b78107d77b1003e2a23586a565d946bf203490031f97658dbd87"} Jul 15 12:30:16.137291 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.137177 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" event={"ID":"23c61466-535d-433d-ba92-d9eaf1684a9e","Type":"ContainerStarted","Data":"94f388b28c09eab79c4aa6b88f13e298c0a9c898cb4bfc45b8a25bae70997324"} Jul 15 12:30:16.138840 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.138792 2559 generic.go:356] "Generic (PLEG): container finished" podID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerID="4ff3645f03b4195fee4db8dd01e808d27ff87c54dee6b1dafa771d1592336004" exitCode=0 Jul 15 12:30:16.138965 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.138894 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" event={"ID":"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f","Type":"ContainerDied","Data":"4ff3645f03b4195fee4db8dd01e808d27ff87c54dee6b1dafa771d1592336004"} Jul 15 12:30:16.140602 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.140579 2559 generic.go:356] "Generic (PLEG): container finished" podID="914c99bc-2302-4dc1-bf11-028d24766371" containerID="a1eb22d6973bb38999364def043ed9bada1ca4cf817c521fcc961cb88e8414a3" exitCode=0 Jul 15 12:30:16.140704 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.140661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" event={"ID":"914c99bc-2302-4dc1-bf11-028d24766371","Type":"ContainerDied","Data":"a1eb22d6973bb38999364def043ed9bada1ca4cf817c521fcc961cb88e8414a3"} Jul 15 12:30:16.140755 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.140719 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" event={"ID":"914c99bc-2302-4dc1-bf11-028d24766371","Type":"ContainerStarted","Data":"0f7d16feb0b9efb03502ad0cfc9a8e0cb384d712e2c6957657fd1f8f3dbba2a5"} Jul 15 12:30:16.142816 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.142704 2559 generic.go:356] "Generic (PLEG): container finished" podID="5596d224-0d77-437a-ad3d-971e8c95b127" containerID="d310915f2b2e03e8c23eb3959b3957cfacc228447e35fa6a1dfd4a48de9e3a6c" exitCode=0 Jul 15 12:30:16.142816 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:16.142760 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" event={"ID":"5596d224-0d77-437a-ad3d-971e8c95b127","Type":"ContainerDied","Data":"d310915f2b2e03e8c23eb3959b3957cfacc228447e35fa6a1dfd4a48de9e3a6c"} Jul 15 12:30:17.150542 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:17.150513 2559 generic.go:356] "Generic (PLEG): container finished" podID="5596d224-0d77-437a-ad3d-971e8c95b127" containerID="91168ddcb70bd1acacd5e8d297f3b4f0d0e7995f12ccb493e270f9be93548c5f" exitCode=0 Jul 15 12:30:17.150917 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:17.150651 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" event={"ID":"5596d224-0d77-437a-ad3d-971e8c95b127","Type":"ContainerDied","Data":"91168ddcb70bd1acacd5e8d297f3b4f0d0e7995f12ccb493e270f9be93548c5f"} Jul 15 12:30:17.152668 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:17.152646 2559 generic.go:356] "Generic (PLEG): container finished" podID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerID="a9bf455a7b89d4f942586f306023c685e4ceddccfd950bad206942403e2b7323" exitCode=0 Jul 15 12:30:17.152744 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:17.152725 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" event={"ID":"23c61466-535d-433d-ba92-d9eaf1684a9e","Type":"ContainerDied","Data":"a9bf455a7b89d4f942586f306023c685e4ceddccfd950bad206942403e2b7323"} Jul 15 12:30:18.164726 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.164694 2559 generic.go:356] "Generic (PLEG): container finished" podID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerID="7402e5cf3ea358f9e9937698ac16b06bfba8c281301adeed30f8c36bd350ed5d" exitCode=0 Jul 15 12:30:18.165168 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.164768 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" event={"ID":"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f","Type":"ContainerDied","Data":"7402e5cf3ea358f9e9937698ac16b06bfba8c281301adeed30f8c36bd350ed5d"} Jul 15 12:30:18.166737 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.166701 2559 generic.go:356] "Generic (PLEG): container finished" podID="914c99bc-2302-4dc1-bf11-028d24766371" containerID="73a95642b26e1661d5a86b0febd25ea84d3729e4ca5c824f7de1223fc6c691e6" exitCode=0 Jul 15 12:30:18.166847 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.166778 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" event={"ID":"914c99bc-2302-4dc1-bf11-028d24766371","Type":"ContainerDied","Data":"73a95642b26e1661d5a86b0febd25ea84d3729e4ca5c824f7de1223fc6c691e6"} Jul 15 12:30:18.168993 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.168976 2559 generic.go:356] "Generic (PLEG): container finished" podID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerID="021fb7619c9c0575226e7e9b7b2216ba398417fa7fa1cc32f3d05646d0ac011d" exitCode=0 Jul 15 12:30:18.169107 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.169032 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" event={"ID":"23c61466-535d-433d-ba92-d9eaf1684a9e","Type":"ContainerDied","Data":"021fb7619c9c0575226e7e9b7b2216ba398417fa7fa1cc32f3d05646d0ac011d"} Jul 15 12:30:18.311303 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.311276 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:18.412150 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.412122 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-bundle\") pod \"5596d224-0d77-437a-ad3d-971e8c95b127\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " Jul 15 12:30:18.412271 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.412177 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-util\") pod \"5596d224-0d77-437a-ad3d-971e8c95b127\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " Jul 15 12:30:18.412271 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.412250 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt5ds\" (UniqueName: \"kubernetes.io/projected/5596d224-0d77-437a-ad3d-971e8c95b127-kube-api-access-dt5ds\") pod \"5596d224-0d77-437a-ad3d-971e8c95b127\" (UID: \"5596d224-0d77-437a-ad3d-971e8c95b127\") " Jul 15 12:30:18.412862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.412829 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-bundle" (OuterVolumeSpecName: "bundle") pod "5596d224-0d77-437a-ad3d-971e8c95b127" (UID: "5596d224-0d77-437a-ad3d-971e8c95b127"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:18.414403 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.414369 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5596d224-0d77-437a-ad3d-971e8c95b127-kube-api-access-dt5ds" (OuterVolumeSpecName: "kube-api-access-dt5ds") pod "5596d224-0d77-437a-ad3d-971e8c95b127" (UID: "5596d224-0d77-437a-ad3d-971e8c95b127"). InnerVolumeSpecName "kube-api-access-dt5ds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:30:18.417871 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.417823 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-util" (OuterVolumeSpecName: "util") pod "5596d224-0d77-437a-ad3d-971e8c95b127" (UID: "5596d224-0d77-437a-ad3d-971e8c95b127"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:18.512959 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.512921 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt5ds\" (UniqueName: \"kubernetes.io/projected/5596d224-0d77-437a-ad3d-971e8c95b127-kube-api-access-dt5ds\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:18.512959 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.512955 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:18.512959 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:18.512966 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5596d224-0d77-437a-ad3d-971e8c95b127-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:19.175599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.175565 2559 generic.go:356] "Generic (PLEG): container finished" podID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerID="bcf91783ede15a4b69e5c1930cc3c4a145c248d97bb1b35a7993aef1ae9017c0" exitCode=0 Jul 15 12:30:19.176025 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.175658 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" event={"ID":"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f","Type":"ContainerDied","Data":"bcf91783ede15a4b69e5c1930cc3c4a145c248d97bb1b35a7993aef1ae9017c0"} Jul 15 12:30:19.177739 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.177715 2559 generic.go:356] "Generic (PLEG): container finished" podID="914c99bc-2302-4dc1-bf11-028d24766371" containerID="c2a65d196806e0d2095db68c54469e49b26cf059d78e6ed1ffc4af9f37ca0407" exitCode=0 Jul 15 12:30:19.177850 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.177776 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" event={"ID":"914c99bc-2302-4dc1-bf11-028d24766371","Type":"ContainerDied","Data":"c2a65d196806e0d2095db68c54469e49b26cf059d78e6ed1ffc4af9f37ca0407"} Jul 15 12:30:19.179473 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.179451 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" event={"ID":"5596d224-0d77-437a-ad3d-971e8c95b127","Type":"ContainerDied","Data":"1e975ccd2be734b4745531fda9feefee89d4f52bb17cb802571dd372004a68bd"} Jul 15 12:30:19.179556 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.179488 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e975ccd2be734b4745531fda9feefee89d4f52bb17cb802571dd372004a68bd" Jul 15 12:30:19.179556 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.179493 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6" Jul 15 12:30:19.315484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.315462 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:19.421267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.421207 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv89r\" (UniqueName: \"kubernetes.io/projected/23c61466-535d-433d-ba92-d9eaf1684a9e-kube-api-access-zv89r\") pod \"23c61466-535d-433d-ba92-d9eaf1684a9e\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " Jul 15 12:30:19.421267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.421281 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-bundle\") pod \"23c61466-535d-433d-ba92-d9eaf1684a9e\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " Jul 15 12:30:19.421515 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.421317 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-util\") pod \"23c61466-535d-433d-ba92-d9eaf1684a9e\" (UID: \"23c61466-535d-433d-ba92-d9eaf1684a9e\") " Jul 15 12:30:19.421797 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.421771 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-bundle" (OuterVolumeSpecName: "bundle") pod "23c61466-535d-433d-ba92-d9eaf1684a9e" (UID: "23c61466-535d-433d-ba92-d9eaf1684a9e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:19.423410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.423384 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c61466-535d-433d-ba92-d9eaf1684a9e-kube-api-access-zv89r" (OuterVolumeSpecName: "kube-api-access-zv89r") pod "23c61466-535d-433d-ba92-d9eaf1684a9e" (UID: "23c61466-535d-433d-ba92-d9eaf1684a9e"). InnerVolumeSpecName "kube-api-access-zv89r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:30:19.426670 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.426621 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-util" (OuterVolumeSpecName: "util") pod "23c61466-535d-433d-ba92-d9eaf1684a9e" (UID: "23c61466-535d-433d-ba92-d9eaf1684a9e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:19.521976 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.521940 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv89r\" (UniqueName: \"kubernetes.io/projected/23c61466-535d-433d-ba92-d9eaf1684a9e-kube-api-access-zv89r\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:19.521976 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.521971 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:19.521976 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:19.521979 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23c61466-535d-433d-ba92-d9eaf1684a9e-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:20.186284 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.186255 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" Jul 15 12:30:20.186284 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.186277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2" event={"ID":"23c61466-535d-433d-ba92-d9eaf1684a9e","Type":"ContainerDied","Data":"94f388b28c09eab79c4aa6b88f13e298c0a9c898cb4bfc45b8a25bae70997324"} Jul 15 12:30:20.186694 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.186324 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94f388b28c09eab79c4aa6b88f13e298c0a9c898cb4bfc45b8a25bae70997324" Jul 15 12:30:20.320423 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.320394 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:20.329299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.329276 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-util\") pod \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " Jul 15 12:30:20.329395 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.329352 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gkcz\" (UniqueName: \"kubernetes.io/projected/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-kube-api-access-2gkcz\") pod \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " Jul 15 12:30:20.329395 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.329385 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-bundle\") pod \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\" (UID: \"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f\") " Jul 15 12:30:20.329862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.329837 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-bundle" (OuterVolumeSpecName: "bundle") pod "27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" (UID: "27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:20.331366 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.331344 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-kube-api-access-2gkcz" (OuterVolumeSpecName: "kube-api-access-2gkcz") pod "27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" (UID: "27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f"). InnerVolumeSpecName "kube-api-access-2gkcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:30:20.334148 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.334127 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-util" (OuterVolumeSpecName: "util") pod "27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" (UID: "27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:20.349354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.349331 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:20.430360 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430288 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-util\") pod \"914c99bc-2302-4dc1-bf11-028d24766371\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " Jul 15 12:30:20.430360 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430349 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-bundle\") pod \"914c99bc-2302-4dc1-bf11-028d24766371\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " Jul 15 12:30:20.430520 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430402 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74bn4\" (UniqueName: \"kubernetes.io/projected/914c99bc-2302-4dc1-bf11-028d24766371-kube-api-access-74bn4\") pod \"914c99bc-2302-4dc1-bf11-028d24766371\" (UID: \"914c99bc-2302-4dc1-bf11-028d24766371\") " Jul 15 12:30:20.430606 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430590 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:20.430642 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430613 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gkcz\" (UniqueName: \"kubernetes.io/projected/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-kube-api-access-2gkcz\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:20.430642 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430628 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:20.430909 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.430876 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-bundle" (OuterVolumeSpecName: "bundle") pod "914c99bc-2302-4dc1-bf11-028d24766371" (UID: "914c99bc-2302-4dc1-bf11-028d24766371"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:20.432432 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.432411 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914c99bc-2302-4dc1-bf11-028d24766371-kube-api-access-74bn4" (OuterVolumeSpecName: "kube-api-access-74bn4") pod "914c99bc-2302-4dc1-bf11-028d24766371" (UID: "914c99bc-2302-4dc1-bf11-028d24766371"). InnerVolumeSpecName "kube-api-access-74bn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:30:20.435778 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.435744 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-util" (OuterVolumeSpecName: "util") pod "914c99bc-2302-4dc1-bf11-028d24766371" (UID: "914c99bc-2302-4dc1-bf11-028d24766371"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:30:20.531299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.531265 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:20.531299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.531292 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74bn4\" (UniqueName: \"kubernetes.io/projected/914c99bc-2302-4dc1-bf11-028d24766371-kube-api-access-74bn4\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:20.531299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:20.531301 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914c99bc-2302-4dc1-bf11-028d24766371-util\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:30:21.193059 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:21.193033 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" Jul 15 12:30:21.193059 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:21.193036 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76" event={"ID":"27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f","Type":"ContainerDied","Data":"88a9b373cc313fce027328551dd73a8887f417891038f694d9f20ff01d294d79"} Jul 15 12:30:21.193518 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:21.193090 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88a9b373cc313fce027328551dd73a8887f417891038f694d9f20ff01d294d79" Jul 15 12:30:21.194893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:21.194872 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" Jul 15 12:30:21.195009 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:21.194897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg" event={"ID":"914c99bc-2302-4dc1-bf11-028d24766371","Type":"ContainerDied","Data":"0f7d16feb0b9efb03502ad0cfc9a8e0cb384d712e2c6957657fd1f8f3dbba2a5"} Jul 15 12:30:21.195009 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:21.194936 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f7d16feb0b9efb03502ad0cfc9a8e0cb384d712e2c6957657fd1f8f3dbba2a5" Jul 15 12:30:40.246390 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246349 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-58dzq"] Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246706 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="util" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246717 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="util" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246740 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="pull" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246746 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="pull" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246751 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="extract" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246757 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="extract" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246765 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="pull" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246770 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="pull" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246778 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="util" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246783 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="util" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246790 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="extract" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246797 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="extract" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246820 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="extract" Jul 15 12:30:40.246819 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246825 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="extract" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246833 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="util" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246838 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="util" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246848 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="pull" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246854 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="pull" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246862 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="extract" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246866 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="extract" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246874 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="util" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246878 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="util" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246883 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="pull" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246888 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="pull" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246941 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="23c61466-535d-433d-ba92-d9eaf1684a9e" containerName="extract" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246946 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f" containerName="extract" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246954 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5596d224-0d77-437a-ad3d-971e8c95b127" containerName="extract" Jul 15 12:30:40.247267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.246962 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="914c99bc-2302-4dc1-bf11-028d24766371" containerName="extract" Jul 15 12:30:40.255072 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.255049 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:40.258387 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.258361 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bb6dx\"" Jul 15 12:30:40.260490 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.260462 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-58dzq"] Jul 15 12:30:40.395311 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.395278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpq7m\" (UniqueName: \"kubernetes.io/projected/9709b468-dc18-488e-8931-3f51e1318abc-kube-api-access-cpq7m\") pod \"authorino-operator-75cc8d5cb7-58dzq\" (UID: \"9709b468-dc18-488e-8931-3f51e1318abc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:40.496487 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.496401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpq7m\" (UniqueName: \"kubernetes.io/projected/9709b468-dc18-488e-8931-3f51e1318abc-kube-api-access-cpq7m\") pod \"authorino-operator-75cc8d5cb7-58dzq\" (UID: \"9709b468-dc18-488e-8931-3f51e1318abc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:40.506977 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.506945 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpq7m\" (UniqueName: \"kubernetes.io/projected/9709b468-dc18-488e-8931-3f51e1318abc-kube-api-access-cpq7m\") pod \"authorino-operator-75cc8d5cb7-58dzq\" (UID: \"9709b468-dc18-488e-8931-3f51e1318abc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:40.566508 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.566477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:40.720774 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:40.720753 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-58dzq"] Jul 15 12:30:40.722793 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:40.722771 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9709b468_dc18_488e_8931_3f51e1318abc.slice/crio-c9b931025d01274dfea83936a4632d5b2cd8b6230e816cf90b1111355c52ea0b WatchSource:0}: Error finding container c9b931025d01274dfea83936a4632d5b2cd8b6230e816cf90b1111355c52ea0b: Status 404 returned error can't find the container with id c9b931025d01274dfea83936a4632d5b2cd8b6230e816cf90b1111355c52ea0b Jul 15 12:30:41.301780 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:41.301740 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" event={"ID":"9709b468-dc18-488e-8931-3f51e1318abc","Type":"ContainerStarted","Data":"c9b931025d01274dfea83936a4632d5b2cd8b6230e816cf90b1111355c52ea0b"} Jul 15 12:30:43.316560 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:43.316515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" event={"ID":"9709b468-dc18-488e-8931-3f51e1318abc","Type":"ContainerStarted","Data":"8e03966a199e4618b33ed53b121086a881881ff8cc95ef7e023ef871e216d491"} Jul 15 12:30:43.316957 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:43.316609 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:43.348036 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:43.347982 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" podStartSLOduration=1.220886201 podStartE2EDuration="3.347969352s" totalImagesPullingTime="2.127083151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:43.344563819 +0000 UTC m=+443.895323350" watchObservedRunningTime="2026-07-15 12:30:43.347969352 +0000 UTC m=+443.898728883" Jul 15 12:30:45.573207 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.573175 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm"] Jul 15 12:30:45.576674 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.576656 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.579492 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.579474 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mkb9k\"" Jul 15 12:30:45.589596 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.589572 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm"] Jul 15 12:30:45.637156 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.637131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5fqx\" (UniqueName: \"kubernetes.io/projected/8022ad91-6be1-42bc-b12b-d558e8451207-kube-api-access-b5fqx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.637257 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.637179 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8022ad91-6be1-42bc-b12b-d558e8451207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.737893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.737869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5fqx\" (UniqueName: \"kubernetes.io/projected/8022ad91-6be1-42bc-b12b-d558e8451207-kube-api-access-b5fqx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.738003 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.737910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8022ad91-6be1-42bc-b12b-d558e8451207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.738250 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.738235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8022ad91-6be1-42bc-b12b-d558e8451207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.747182 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.747157 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5fqx\" (UniqueName: \"kubernetes.io/projected/8022ad91-6be1-42bc-b12b-d558e8451207-kube-api-access-b5fqx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:45.886762 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:45.886702 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:46.026199 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:46.026165 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8022ad91_6be1_42bc_b12b_d558e8451207.slice/crio-e53b46b65e67fe97c39f161d09769a82beb5d6868e3e5f5813fd84ed0f92a3b2 WatchSource:0}: Error finding container e53b46b65e67fe97c39f161d09769a82beb5d6868e3e5f5813fd84ed0f92a3b2: Status 404 returned error can't find the container with id e53b46b65e67fe97c39f161d09769a82beb5d6868e3e5f5813fd84ed0f92a3b2 Jul 15 12:30:46.030651 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:46.030622 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm"] Jul 15 12:30:46.333969 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:46.333941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" event={"ID":"8022ad91-6be1-42bc-b12b-d558e8451207","Type":"ContainerStarted","Data":"e53b46b65e67fe97c39f161d09769a82beb5d6868e3e5f5813fd84ed0f92a3b2"} Jul 15 12:30:48.862402 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:48.862370 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2"] Jul 15 12:30:48.865917 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:48.865891 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:30:48.887051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:48.887029 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jzn77\"" Jul 15 12:30:48.887213 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:48.887072 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 12:30:48.908084 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:48.908058 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2"] Jul 15 12:30:48.962397 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:48.962365 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhjhh\" (UniqueName: \"kubernetes.io/projected/8ff0870c-aa56-4af7-b340-28e9470df6ae-kube-api-access-zhjhh\") pod \"dns-operator-controller-manager-864dd78c4c-6rbm2\" (UID: \"8ff0870c-aa56-4af7-b340-28e9470df6ae\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:30:49.062976 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:49.062942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhjhh\" (UniqueName: \"kubernetes.io/projected/8ff0870c-aa56-4af7-b340-28e9470df6ae-kube-api-access-zhjhh\") pod \"dns-operator-controller-manager-864dd78c4c-6rbm2\" (UID: \"8ff0870c-aa56-4af7-b340-28e9470df6ae\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:30:49.090661 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:49.090629 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhjhh\" (UniqueName: \"kubernetes.io/projected/8ff0870c-aa56-4af7-b340-28e9470df6ae-kube-api-access-zhjhh\") pod \"dns-operator-controller-manager-864dd78c4c-6rbm2\" (UID: \"8ff0870c-aa56-4af7-b340-28e9470df6ae\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:30:49.176272 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:49.176208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:30:49.357412 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:49.357392 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2"] Jul 15 12:30:50.403102 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:50.403070 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff0870c_aa56_4af7_b340_28e9470df6ae.slice/crio-a99af5f00b6e1ba3370444dde8a9592688928d33d2b85939152712ab7aa5e795 WatchSource:0}: Error finding container a99af5f00b6e1ba3370444dde8a9592688928d33d2b85939152712ab7aa5e795: Status 404 returned error can't find the container with id a99af5f00b6e1ba3370444dde8a9592688928d33d2b85939152712ab7aa5e795 Jul 15 12:30:50.860926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:50.860895 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm"] Jul 15 12:30:50.864548 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:50.864521 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:30:50.867607 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:50.867587 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g466q\"" Jul 15 12:30:50.875909 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:50.875890 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm"] Jul 15 12:30:50.980411 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:50.980381 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs76q\" (UniqueName: \"kubernetes.io/projected/3795f504-1213-42e8-b3e7-203c4e20b952-kube-api-access-vs76q\") pod \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" (UID: \"3795f504-1213-42e8-b3e7-203c4e20b952\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:30:51.081672 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.081645 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs76q\" (UniqueName: \"kubernetes.io/projected/3795f504-1213-42e8-b3e7-203c4e20b952-kube-api-access-vs76q\") pod \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" (UID: \"3795f504-1213-42e8-b3e7-203c4e20b952\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:30:51.091949 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.091923 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs76q\" (UniqueName: \"kubernetes.io/projected/3795f504-1213-42e8-b3e7-203c4e20b952-kube-api-access-vs76q\") pod \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" (UID: \"3795f504-1213-42e8-b3e7-203c4e20b952\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:30:51.176717 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.176593 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:30:51.367077 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.367035 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" event={"ID":"8022ad91-6be1-42bc-b12b-d558e8451207","Type":"ContainerStarted","Data":"972039311b0b5bf0b022202e014e59974d9afd5a6ccfb1ad2633dc344a5b8bba"} Jul 15 12:30:51.367321 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.367299 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:30:51.369192 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.369151 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" event={"ID":"8ff0870c-aa56-4af7-b340-28e9470df6ae","Type":"ContainerStarted","Data":"a99af5f00b6e1ba3370444dde8a9592688928d33d2b85939152712ab7aa5e795"} Jul 15 12:30:51.401562 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.401483 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" podStartSLOduration=1.970146702 podStartE2EDuration="6.401462869s" totalImagesPullingTime="4.431316167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:51.397305254 +0000 UTC m=+451.948064817" watchObservedRunningTime="2026-07-15 12:30:51.401462869 +0000 UTC m=+451.952222403" Jul 15 12:30:51.595750 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:51.595726 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm"] Jul 15 12:30:51.639014 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:30:51.638973 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3795f504_1213_42e8_b3e7_203c4e20b952.slice/crio-ad2fac57e399d13477cb945bb737dbf6ee1b92933690ce0931210492f2e44340 WatchSource:0}: Error finding container ad2fac57e399d13477cb945bb737dbf6ee1b92933690ce0931210492f2e44340: Status 404 returned error can't find the container with id ad2fac57e399d13477cb945bb737dbf6ee1b92933690ce0931210492f2e44340 Jul 15 12:30:52.379867 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:52.379834 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" event={"ID":"3795f504-1213-42e8-b3e7-203c4e20b952","Type":"ContainerStarted","Data":"ad2fac57e399d13477cb945bb737dbf6ee1b92933690ce0931210492f2e44340"} Jul 15 12:30:53.390519 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:53.390480 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" event={"ID":"8ff0870c-aa56-4af7-b340-28e9470df6ae","Type":"ContainerStarted","Data":"9bac2ab484e05ed8c9adc19f8eaf7f56997f262f5a6bf68e9a316f8e3995339b"} Jul 15 12:30:53.390978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:53.390623 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:30:53.421896 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:53.421853 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" podStartSLOduration=3.207548176 podStartE2EDuration="5.421838569s" totalImagesPullingTime="2.214290393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:53.420564955 +0000 UTC m=+453.971324489" watchObservedRunningTime="2026-07-15 12:30:53.421838569 +0000 UTC m=+453.972598100" Jul 15 12:30:54.324753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:54.324722 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-58dzq" Jul 15 12:30:54.397993 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:54.397957 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" event={"ID":"3795f504-1213-42e8-b3e7-203c4e20b952","Type":"ContainerStarted","Data":"f5054adb711531184aebe0df7092893b1a4c4ce7cc99ee7e7130cb84615a87bc"} Jul 15 12:30:54.398386 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:54.398172 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:30:54.421688 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:30:54.421642 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" podStartSLOduration=2.588455787 podStartE2EDuration="4.421626697s" totalImagesPullingTime="1.83317091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:54.420663889 +0000 UTC m=+454.971423421" watchObservedRunningTime="2026-07-15 12:30:54.421626697 +0000 UTC m=+454.972386228" Jul 15 12:31:02.382993 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:02.382959 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.094057 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.094010 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f"] Jul 15 12:31:04.098258 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.098229 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.117255 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.117219 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f"] Jul 15 12:31:04.193309 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.193281 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kxxp\" (UniqueName: \"kubernetes.io/projected/94044506-03ef-4c07-a57c-e59b5a0adaeb-kube-api-access-2kxxp\") pod \"kuadrant-operator-controller-manager-86878c769f-95b7f\" (UID: \"94044506-03ef-4c07-a57c-e59b5a0adaeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.193446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.193337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94044506-03ef-4c07-a57c-e59b5a0adaeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-95b7f\" (UID: \"94044506-03ef-4c07-a57c-e59b5a0adaeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.208973 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.208946 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm"] Jul 15 12:31:04.209182 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.209139 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" containerName="manager" containerID="cri-o://972039311b0b5bf0b022202e014e59974d9afd5a6ccfb1ad2633dc344a5b8bba" gracePeriod=2 Jul 15 12:31:04.219008 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.218973 2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8022ad91-6be1-42bc-b12b-d558e8451207\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\": pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.219208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.219183 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm"] Jul 15 12:31:04.238475 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.238448 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f"] Jul 15 12:31:04.238792 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.238767 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-2kxxp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" Jul 15 12:31:04.242791 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.242770 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf"] Jul 15 12:31:04.243167 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.243147 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" containerName="manager" Jul 15 12:31:04.243167 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.243165 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" containerName="manager" Jul 15 12:31:04.243300 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.243234 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" containerName="manager" Jul 15 12:31:04.246187 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.246170 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f"] Jul 15 12:31:04.246264 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.246250 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.256041 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.256019 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf"] Jul 15 12:31:04.261356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.261333 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm"] Jul 15 12:31:04.261547 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.261501 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" containerName="manager" containerID="cri-o://f5054adb711531184aebe0df7092893b1a4c4ce7cc99ee7e7130cb84615a87bc" gracePeriod=2 Jul 15 12:31:04.263550 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.263528 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.276166 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.276138 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm"] Jul 15 12:31:04.280321 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.279310 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h"] Jul 15 12:31:04.280321 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.279902 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" containerName="manager" Jul 15 12:31:04.280321 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.279918 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" containerName="manager" Jul 15 12:31:04.280321 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.280031 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" containerName="manager" Jul 15 12:31:04.281255 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.281224 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.283192 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.283176 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.283413 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.283394 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.285716 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.285694 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.287651 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.287628 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.294082 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.294059 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kxxp\" (UniqueName: \"kubernetes.io/projected/94044506-03ef-4c07-a57c-e59b5a0adaeb-kube-api-access-2kxxp\") pod \"kuadrant-operator-controller-manager-86878c769f-95b7f\" (UID: \"94044506-03ef-4c07-a57c-e59b5a0adaeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.294188 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.294110 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94044506-03ef-4c07-a57c-e59b5a0adaeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-95b7f\" (UID: \"94044506-03ef-4c07-a57c-e59b5a0adaeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.294558 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.294534 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94044506-03ef-4c07-a57c-e59b5a0adaeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-95b7f\" (UID: \"94044506-03ef-4c07-a57c-e59b5a0adaeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.297926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.297903 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h"] Jul 15 12:31:04.301227 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.301209 2559 projected.go:196] Error preparing data for projected volume kube-api-access-2kxxp for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-95b7f" not found Jul 15 12:31:04.301330 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.301282 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94044506-03ef-4c07-a57c-e59b5a0adaeb-kube-api-access-2kxxp podName:94044506-03ef-4c07-a57c-e59b5a0adaeb nodeName:}" failed. No retries permitted until 2026-07-15 12:31:04.801266853 +0000 UTC m=+465.352026362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2kxxp" (UniqueName: "kubernetes.io/projected/94044506-03ef-4c07-a57c-e59b5a0adaeb-kube-api-access-2kxxp") pod "kuadrant-operator-controller-manager-86878c769f-95b7f" (UID: "94044506-03ef-4c07-a57c-e59b5a0adaeb") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-95b7f" not found Jul 15 12:31:04.319656 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.319631 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.395140 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.395102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kfq5h\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.395266 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.395222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/469ee67f-2d57-478b-b08a-b70a91eeadb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxgbf\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.395266 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.395255 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pg9z\" (UniqueName: \"kubernetes.io/projected/469ee67f-2d57-478b-b08a-b70a91eeadb2-kube-api-access-8pg9z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxgbf\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.395358 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.395282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-849cg\" (UniqueName: \"kubernetes.io/projected/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-kube-api-access-849cg\") pod \"kuadrant-operator-controller-manager-86878c769f-kfq5h\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.401021 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.400995 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6rbm2" Jul 15 12:31:04.422309 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.422275 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.424419 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.424389 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.463402 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.463373 2559 generic.go:356] "Generic (PLEG): container finished" podID="3795f504-1213-42e8-b3e7-203c4e20b952" containerID="f5054adb711531184aebe0df7092893b1a4c4ce7cc99ee7e7130cb84615a87bc" exitCode=0 Jul 15 12:31:04.465575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.465546 2559 generic.go:356] "Generic (PLEG): container finished" podID="8022ad91-6be1-42bc-b12b-d558e8451207" containerID="972039311b0b5bf0b022202e014e59974d9afd5a6ccfb1ad2633dc344a5b8bba" exitCode=0 Jul 15 12:31:04.465703 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.465679 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.468429 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.468402 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.472066 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.471781 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.474346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.474231 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.474551 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.474523 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.477003 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.476938 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.479138 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.479117 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.481198 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.481177 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.496563 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.496541 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kfq5h\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.496684 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.496587 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/469ee67f-2d57-478b-b08a-b70a91eeadb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxgbf\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.496684 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.496608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pg9z\" (UniqueName: \"kubernetes.io/projected/469ee67f-2d57-478b-b08a-b70a91eeadb2-kube-api-access-8pg9z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxgbf\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.496684 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.496640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-849cg\" (UniqueName: \"kubernetes.io/projected/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-kube-api-access-849cg\") pod \"kuadrant-operator-controller-manager-86878c769f-kfq5h\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.497174 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.497002 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kfq5h\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.497462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.497370 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/469ee67f-2d57-478b-b08a-b70a91eeadb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxgbf\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.511043 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.511022 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-849cg\" (UniqueName: \"kubernetes.io/projected/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-kube-api-access-849cg\") pod \"kuadrant-operator-controller-manager-86878c769f-kfq5h\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.511166 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.511147 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pg9z\" (UniqueName: \"kubernetes.io/projected/469ee67f-2d57-478b-b08a-b70a91eeadb2-kube-api-access-8pg9z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxgbf\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.540347 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.540325 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.542700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.542681 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.542996 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.542975 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.545417 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.545398 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.548367 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.548348 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.551112 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.551093 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:04.553517 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.553499 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:04.555937 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:04.555918 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:04.597420 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.597398 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94044506-03ef-4c07-a57c-e59b5a0adaeb-extensions-socket-volume\") pod \"94044506-03ef-4c07-a57c-e59b5a0adaeb\" (UID: \"94044506-03ef-4c07-a57c-e59b5a0adaeb\") " Jul 15 12:31:04.597590 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.597578 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kxxp\" (UniqueName: \"kubernetes.io/projected/94044506-03ef-4c07-a57c-e59b5a0adaeb-kube-api-access-2kxxp\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:04.597645 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.597626 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94044506-03ef-4c07-a57c-e59b5a0adaeb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "94044506-03ef-4c07-a57c-e59b5a0adaeb" (UID: "94044506-03ef-4c07-a57c-e59b5a0adaeb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:31:04.655698 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.655649 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:04.663386 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.663367 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:04.698584 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.698553 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5fqx\" (UniqueName: \"kubernetes.io/projected/8022ad91-6be1-42bc-b12b-d558e8451207-kube-api-access-b5fqx\") pod \"8022ad91-6be1-42bc-b12b-d558e8451207\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " Jul 15 12:31:04.698712 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.698649 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8022ad91-6be1-42bc-b12b-d558e8451207-extensions-socket-volume\") pod \"8022ad91-6be1-42bc-b12b-d558e8451207\" (UID: \"8022ad91-6be1-42bc-b12b-d558e8451207\") " Jul 15 12:31:04.698712 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.698688 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs76q\" (UniqueName: \"kubernetes.io/projected/3795f504-1213-42e8-b3e7-203c4e20b952-kube-api-access-vs76q\") pod \"3795f504-1213-42e8-b3e7-203c4e20b952\" (UID: \"3795f504-1213-42e8-b3e7-203c4e20b952\") " Jul 15 12:31:04.698869 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.698841 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94044506-03ef-4c07-a57c-e59b5a0adaeb-extensions-socket-volume\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:04.699395 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.699361 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8022ad91-6be1-42bc-b12b-d558e8451207-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8022ad91-6be1-42bc-b12b-d558e8451207" (UID: "8022ad91-6be1-42bc-b12b-d558e8451207"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:31:04.700786 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.700755 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3795f504-1213-42e8-b3e7-203c4e20b952-kube-api-access-vs76q" (OuterVolumeSpecName: "kube-api-access-vs76q") pod "3795f504-1213-42e8-b3e7-203c4e20b952" (UID: "3795f504-1213-42e8-b3e7-203c4e20b952"). InnerVolumeSpecName "kube-api-access-vs76q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:31:04.700786 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.700766 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8022ad91-6be1-42bc-b12b-d558e8451207-kube-api-access-b5fqx" (OuterVolumeSpecName: "kube-api-access-b5fqx") pod "8022ad91-6be1-42bc-b12b-d558e8451207" (UID: "8022ad91-6be1-42bc-b12b-d558e8451207"). InnerVolumeSpecName "kube-api-access-b5fqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:31:04.799272 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.799246 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8022ad91-6be1-42bc-b12b-d558e8451207-extensions-socket-volume\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:04.799272 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.799272 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs76q\" (UniqueName: \"kubernetes.io/projected/3795f504-1213-42e8-b3e7-203c4e20b952-kube-api-access-vs76q\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:04.799428 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.799280 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5fqx\" (UniqueName: \"kubernetes.io/projected/8022ad91-6be1-42bc-b12b-d558e8451207-kube-api-access-b5fqx\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:04.805656 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:04.805633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf"] Jul 15 12:31:04.807239 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:04.807213 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469ee67f_2d57_478b_b08a_b70a91eeadb2.slice/crio-1d639bfa2a7fdbf352b1173166cd6dc24274cb06bd460dbf5030fcf6bcf97dd8 WatchSource:0}: Error finding container 1d639bfa2a7fdbf352b1173166cd6dc24274cb06bd460dbf5030fcf6bcf97dd8: Status 404 returned error can't find the container with id 1d639bfa2a7fdbf352b1173166cd6dc24274cb06bd460dbf5030fcf6bcf97dd8 Jul 15 12:31:05.049722 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.049693 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h"] Jul 15 12:31:05.050878 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:05.050850 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee6d13d_3ddd_4cd0_bb56_5db5250a8ae5.slice/crio-1ec459b90ed7792fc849208f7b59c42ef1fc207f9903684dab658396b102d82a WatchSource:0}: Error finding container 1ec459b90ed7792fc849208f7b59c42ef1fc207f9903684dab658396b102d82a: Status 404 returned error can't find the container with id 1ec459b90ed7792fc849208f7b59c42ef1fc207f9903684dab658396b102d82a Jul 15 12:31:05.473122 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.473096 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:05.473536 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.473143 2559 scope.go:117] "RemoveContainer" containerID="972039311b0b5bf0b022202e014e59974d9afd5a6ccfb1ad2633dc344a5b8bba" Jul 15 12:31:05.474566 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.474528 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:05.475878 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.475845 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:05.476721 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.476691 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" event={"ID":"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5","Type":"ContainerStarted","Data":"697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee"} Jul 15 12:31:05.476846 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.476744 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" event={"ID":"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5","Type":"ContainerStarted","Data":"1ec459b90ed7792fc849208f7b59c42ef1fc207f9903684dab658396b102d82a"} Jul 15 12:31:05.476846 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.476770 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:05.478358 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.478329 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:05.479014 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.478794 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:05.479014 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.478876 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" event={"ID":"469ee67f-2d57-478b-b08a-b70a91eeadb2","Type":"ContainerStarted","Data":"b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785"} Jul 15 12:31:05.479014 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.478919 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" event={"ID":"469ee67f-2d57-478b-b08a-b70a91eeadb2","Type":"ContainerStarted","Data":"1d639bfa2a7fdbf352b1173166cd6dc24274cb06bd460dbf5030fcf6bcf97dd8"} Jul 15 12:31:05.480508 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.480487 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:05.482749 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.482728 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:05.484727 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.484651 2559 scope.go:117] "RemoveContainer" containerID="f5054adb711531184aebe0df7092893b1a4c4ce7cc99ee7e7130cb84615a87bc" Jul 15 12:31:05.485070 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.484948 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:05.508768 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.508724 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" podStartSLOduration=1.5087122160000002 podStartE2EDuration="1.508712216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:05.506068068 +0000 UTC m=+466.056827621" watchObservedRunningTime="2026-07-15 12:31:05.508712216 +0000 UTC m=+466.059471747" Jul 15 12:31:05.528926 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:05.528890 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" podStartSLOduration=1.528878397 podStartE2EDuration="1.528878397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:05.526828393 +0000 UTC m=+466.077587953" watchObservedRunningTime="2026-07-15 12:31:05.528878397 +0000 UTC m=+466.079637992" Jul 15 12:31:05.529168 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.529146 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:05.531498 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.531479 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-95b7f\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-95b7f" Jul 15 12:31:05.533609 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.533589 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kqbsm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kqbsm" Jul 15 12:31:05.535683 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:05.535659 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pjbxm\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pjbxm" Jul 15 12:31:06.110191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:06.110149 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3795f504-1213-42e8-b3e7-203c4e20b952" path="/var/lib/kubelet/pods/3795f504-1213-42e8-b3e7-203c4e20b952/volumes" Jul 15 12:31:06.110573 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:06.110552 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8022ad91-6be1-42bc-b12b-d558e8451207" path="/var/lib/kubelet/pods/8022ad91-6be1-42bc-b12b-d558e8451207/volumes" Jul 15 12:31:06.110950 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:06.110929 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94044506-03ef-4c07-a57c-e59b5a0adaeb" path="/var/lib/kubelet/pods/94044506-03ef-4c07-a57c-e59b5a0adaeb/volumes" Jul 15 12:31:06.492106 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:06.492074 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:16.494692 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.494661 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:31:16.556641 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.555969 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf"] Jul 15 12:31:16.556641 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.556303 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" podUID="469ee67f-2d57-478b-b08a-b70a91eeadb2" containerName="manager" containerID="cri-o://b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785" gracePeriod=10 Jul 15 12:31:16.564021 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.563995 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:16.809391 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.809368 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:16.997445 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.997416 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pg9z\" (UniqueName: \"kubernetes.io/projected/469ee67f-2d57-478b-b08a-b70a91eeadb2-kube-api-access-8pg9z\") pod \"469ee67f-2d57-478b-b08a-b70a91eeadb2\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " Jul 15 12:31:16.997623 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.997606 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/469ee67f-2d57-478b-b08a-b70a91eeadb2-extensions-socket-volume\") pod \"469ee67f-2d57-478b-b08a-b70a91eeadb2\" (UID: \"469ee67f-2d57-478b-b08a-b70a91eeadb2\") " Jul 15 12:31:16.997983 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.997962 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/469ee67f-2d57-478b-b08a-b70a91eeadb2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "469ee67f-2d57-478b-b08a-b70a91eeadb2" (UID: "469ee67f-2d57-478b-b08a-b70a91eeadb2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:31:16.999423 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:16.999394 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469ee67f-2d57-478b-b08a-b70a91eeadb2-kube-api-access-8pg9z" (OuterVolumeSpecName: "kube-api-access-8pg9z") pod "469ee67f-2d57-478b-b08a-b70a91eeadb2" (UID: "469ee67f-2d57-478b-b08a-b70a91eeadb2"). InnerVolumeSpecName "kube-api-access-8pg9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:31:17.099092 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.099020 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pg9z\" (UniqueName: \"kubernetes.io/projected/469ee67f-2d57-478b-b08a-b70a91eeadb2-kube-api-access-8pg9z\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:17.099092 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.099042 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/469ee67f-2d57-478b-b08a-b70a91eeadb2-extensions-socket-volume\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:17.551904 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.551866 2559 generic.go:356] "Generic (PLEG): container finished" podID="469ee67f-2d57-478b-b08a-b70a91eeadb2" containerID="b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785" exitCode=0 Jul 15 12:31:17.552278 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.551916 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" Jul 15 12:31:17.552278 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.551949 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" event={"ID":"469ee67f-2d57-478b-b08a-b70a91eeadb2","Type":"ContainerDied","Data":"b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785"} Jul 15 12:31:17.552278 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.552005 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf" event={"ID":"469ee67f-2d57-478b-b08a-b70a91eeadb2","Type":"ContainerDied","Data":"1d639bfa2a7fdbf352b1173166cd6dc24274cb06bd460dbf5030fcf6bcf97dd8"} Jul 15 12:31:17.552278 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.552028 2559 scope.go:117] "RemoveContainer" containerID="b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785" Jul 15 12:31:17.564130 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.563959 2559 scope.go:117] "RemoveContainer" containerID="b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785" Jul 15 12:31:17.564411 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:17.564393 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785\": container with ID starting with b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785 not found: ID does not exist" containerID="b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785" Jul 15 12:31:17.564484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.564420 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785"} err="failed to get container status \"b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785\": rpc error: code = NotFound desc = could not find container \"b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785\": container with ID starting with b98baf6578ca23844ee6e793014b8b6ef4b495eb6d1681f3200f9d8ec6951785 not found: ID does not exist" Jul 15 12:31:17.598961 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.598937 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf"] Jul 15 12:31:17.613169 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:17.613146 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxgbf"] Jul 15 12:31:18.110751 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:18.110720 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469ee67f-2d57-478b-b08a-b70a91eeadb2" path="/var/lib/kubelet/pods/469ee67f-2d57-478b-b08a-b70a91eeadb2/volumes" Jul 15 12:31:19.928188 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:19.928157 2559 scope.go:117] "RemoveContainer" containerID="8090e32a1fdae69f05e308127b505e5bf3c5726fdc7803a40feaf5d3e11b6118" Jul 15 12:31:31.542839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.542786 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744ff8f984-2lmqx"] Jul 15 12:31:31.543224 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.543209 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469ee67f-2d57-478b-b08a-b70a91eeadb2" containerName="manager" Jul 15 12:31:31.543267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.543225 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="469ee67f-2d57-478b-b08a-b70a91eeadb2" containerName="manager" Jul 15 12:31:31.543318 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.543308 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="469ee67f-2d57-478b-b08a-b70a91eeadb2" containerName="manager" Jul 15 12:31:31.547680 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.547660 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.558304 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.558282 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744ff8f984-2lmqx"] Jul 15 12:31:31.598356 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598334 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-console-config\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.598492 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5fee2079-f9fb-4d20-8c82-85b274a83537-console-serving-cert\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.598492 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598389 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-service-ca\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.598492 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-trusted-ca-bundle\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.598492 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598462 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-oauth-serving-cert\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.598658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598501 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5fee2079-f9fb-4d20-8c82-85b274a83537-console-oauth-config\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.598658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.598537 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbr78\" (UniqueName: \"kubernetes.io/projected/5fee2079-f9fb-4d20-8c82-85b274a83537-kube-api-access-pbr78\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699251 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699221 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbr78\" (UniqueName: \"kubernetes.io/projected/5fee2079-f9fb-4d20-8c82-85b274a83537-kube-api-access-pbr78\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699270 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-console-config\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699318 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5fee2079-f9fb-4d20-8c82-85b274a83537-console-serving-cert\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-service-ca\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699353 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-trusted-ca-bundle\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699392 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-oauth-serving-cert\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.699663 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.699419 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5fee2079-f9fb-4d20-8c82-85b274a83537-console-oauth-config\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.700127 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.700100 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-service-ca\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.700226 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.700121 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-oauth-serving-cert\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.700226 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.700100 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-console-config\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.700396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.700372 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fee2079-f9fb-4d20-8c82-85b274a83537-trusted-ca-bundle\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.701935 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.701913 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5fee2079-f9fb-4d20-8c82-85b274a83537-console-serving-cert\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.701995 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.701956 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5fee2079-f9fb-4d20-8c82-85b274a83537-console-oauth-config\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.708332 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.708315 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbr78\" (UniqueName: \"kubernetes.io/projected/5fee2079-f9fb-4d20-8c82-85b274a83537-kube-api-access-pbr78\") pod \"console-744ff8f984-2lmqx\" (UID: \"5fee2079-f9fb-4d20-8c82-85b274a83537\") " pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:31.858947 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:31.858900 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:32.002615 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:32.002587 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744ff8f984-2lmqx"] Jul 15 12:31:32.003155 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:32.003133 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fee2079_f9fb_4d20_8c82_85b274a83537.slice/crio-70b4808c746ea810f42e67d8cd0a180e81096344648a6611b90833ba70edfa62 WatchSource:0}: Error finding container 70b4808c746ea810f42e67d8cd0a180e81096344648a6611b90833ba70edfa62: Status 404 returned error can't find the container with id 70b4808c746ea810f42e67d8cd0a180e81096344648a6611b90833ba70edfa62 Jul 15 12:31:32.639458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:32.639421 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744ff8f984-2lmqx" event={"ID":"5fee2079-f9fb-4d20-8c82-85b274a83537","Type":"ContainerStarted","Data":"27cedd1f92cda815398e718d093913eec87e00f7494afbd51e960fe11e1ecac4"} Jul 15 12:31:32.639846 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:32.639471 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744ff8f984-2lmqx" event={"ID":"5fee2079-f9fb-4d20-8c82-85b274a83537","Type":"ContainerStarted","Data":"70b4808c746ea810f42e67d8cd0a180e81096344648a6611b90833ba70edfa62"} Jul 15 12:31:32.663632 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:32.663587 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744ff8f984-2lmqx" podStartSLOduration=1.663572479 podStartE2EDuration="1.663572479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:32.661089665 +0000 UTC m=+493.211849197" watchObservedRunningTime="2026-07-15 12:31:32.663572479 +0000 UTC m=+493.214332068" Jul 15 12:31:33.164790 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.164762 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z"] Jul 15 12:31:33.168854 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.168834 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.171689 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:33.171660 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="2bc79424-ac1c-4dd0-9284-610be84542ec" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.171946 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:33.171921 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-default-gateway-openshift-default-dockercfg-tq9kz\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tq9kz\"" type="*v1.Secret" Jul 15 12:31:33.212446 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc79424-ac1c-4dd0-9284-610be84542ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212547 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212554 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212596 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwpsc\" (UniqueName: \"kubernetes.io/projected/2bc79424-ac1c-4dd0-9284-610be84542ec-kube-api-access-kwpsc\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212676 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212630 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212858 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212676 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212858 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212708 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212858 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212763 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.212858 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.212830 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.213989 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.213958 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z"] Jul 15 12:31:33.313957 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.313931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314090 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.313976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314141 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314182 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314154 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc79424-ac1c-4dd0-9284-610be84542ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314232 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314280 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314280 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwpsc\" (UniqueName: \"kubernetes.io/projected/2bc79424-ac1c-4dd0-9284-610be84542ec-kube-api-access-kwpsc\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314433 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314381 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314433 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314398 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314469 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314407 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314671 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.314834 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.314815 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.315362 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.315340 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc79424-ac1c-4dd0-9284-610be84542ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.316426 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.316410 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.316985 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.316966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.329793 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.329764 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwpsc\" (UniqueName: \"kubernetes.io/projected/2bc79424-ac1c-4dd0-9284-610be84542ec-kube-api-access-kwpsc\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:33.330469 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:33.330452 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc79424-ac1c-4dd0-9284-610be84542ec-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j6q6z\" (UID: \"2bc79424-ac1c-4dd0-9284-610be84542ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:34.060838 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.060793 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tq9kz\"" Jul 15 12:31:34.069726 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.069708 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:34.208109 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.208080 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z"] Jul 15 12:31:34.208515 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:34.208493 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc79424_ac1c_4dd0_9284_610be84542ec.slice/crio-054d0f4430aca0d29021087178dbe63962150d22234ec4e6d0fd8579708c9082 WatchSource:0}: Error finding container 054d0f4430aca0d29021087178dbe63962150d22234ec4e6d0fd8579708c9082: Status 404 returned error can't find the container with id 054d0f4430aca0d29021087178dbe63962150d22234ec4e6d0fd8579708c9082 Jul 15 12:31:34.211008 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.210980 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:31:34.211076 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.211044 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:31:34.211114 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.211073 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:31:34.651546 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.651512 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" event={"ID":"2bc79424-ac1c-4dd0-9284-610be84542ec","Type":"ContainerStarted","Data":"b982c10d02d70cdf3b1100cb6b1cdddad604cef3fabb64dd23c09795b829d55a"} Jul 15 12:31:34.651707 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.651559 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" event={"ID":"2bc79424-ac1c-4dd0-9284-610be84542ec","Type":"ContainerStarted","Data":"054d0f4430aca0d29021087178dbe63962150d22234ec4e6d0fd8579708c9082"} Jul 15 12:31:34.680918 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:34.680871 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" podStartSLOduration=1.680855759 podStartE2EDuration="1.680855759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:34.677757313 +0000 UTC m=+495.228516847" watchObservedRunningTime="2026-07-15 12:31:34.680855759 +0000 UTC m=+495.231615284" Jul 15 12:31:35.070326 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:35.070296 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:35.074786 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:35.074765 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:35.656798 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:35.656768 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:35.657755 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:35.657739 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:31:36.571123 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.571076 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-2rtpn"] Jul 15 12:31:36.574680 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.574662 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:36.577464 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.577446 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vgfcm\"" Jul 15 12:31:36.582789 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.582767 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2rtpn"] Jul 15 12:31:36.645008 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.644986 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qp9r\" (UniqueName: \"kubernetes.io/projected/2b528544-111f-499a-8a00-1f951ab0f58d-kube-api-access-9qp9r\") pod \"authorino-58cd955675-2rtpn\" (UID: \"2b528544-111f-499a-8a00-1f951ab0f58d\") " pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:36.697119 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.697095 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qprbn"] Jul 15 12:31:36.700617 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.700600 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:31:36.709451 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.709419 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qprbn"] Jul 15 12:31:36.746193 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.746169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762-kube-api-access-p2mzq\") pod \"authorino-75d556bd77-qprbn\" (UID: \"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762\") " pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:31:36.746286 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.746237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qp9r\" (UniqueName: \"kubernetes.io/projected/2b528544-111f-499a-8a00-1f951ab0f58d-kube-api-access-9qp9r\") pod \"authorino-58cd955675-2rtpn\" (UID: \"2b528544-111f-499a-8a00-1f951ab0f58d\") " pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:36.756437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.756413 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qp9r\" (UniqueName: \"kubernetes.io/projected/2b528544-111f-499a-8a00-1f951ab0f58d-kube-api-access-9qp9r\") pod \"authorino-58cd955675-2rtpn\" (UID: \"2b528544-111f-499a-8a00-1f951ab0f58d\") " pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:36.847618 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.847561 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762-kube-api-access-p2mzq\") pod \"authorino-75d556bd77-qprbn\" (UID: \"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762\") " pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:31:36.856798 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.856778 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762-kube-api-access-p2mzq\") pod \"authorino-75d556bd77-qprbn\" (UID: \"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762\") " pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:31:36.884713 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:36.884690 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:37.010433 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:37.010411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:31:37.023216 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:37.023190 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2rtpn"] Jul 15 12:31:37.024030 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:37.024006 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b528544_111f_499a_8a00_1f951ab0f58d.slice/crio-0451eb53461f56d008e16fd01261accda8d973e90f1af40dc475681b099e7f56 WatchSource:0}: Error finding container 0451eb53461f56d008e16fd01261accda8d973e90f1af40dc475681b099e7f56: Status 404 returned error can't find the container with id 0451eb53461f56d008e16fd01261accda8d973e90f1af40dc475681b099e7f56 Jul 15 12:31:37.146032 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:37.146004 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qprbn"] Jul 15 12:31:37.147354 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:37.147330 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6bfc0cd_74bf_4856_bc0e_9e8f8a61e762.slice/crio-6c1cd7ad2e7657108c7df7f5b39d3b6e72746fef96a4da2997078b9c831512e9 WatchSource:0}: Error finding container 6c1cd7ad2e7657108c7df7f5b39d3b6e72746fef96a4da2997078b9c831512e9: Status 404 returned error can't find the container with id 6c1cd7ad2e7657108c7df7f5b39d3b6e72746fef96a4da2997078b9c831512e9 Jul 15 12:31:37.667980 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:37.667945 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qprbn" event={"ID":"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762","Type":"ContainerStarted","Data":"6c1cd7ad2e7657108c7df7f5b39d3b6e72746fef96a4da2997078b9c831512e9"} Jul 15 12:31:37.669074 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:37.669049 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2rtpn" event={"ID":"2b528544-111f-499a-8a00-1f951ab0f58d","Type":"ContainerStarted","Data":"0451eb53461f56d008e16fd01261accda8d973e90f1af40dc475681b099e7f56"} Jul 15 12:31:39.718427 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.718389 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:39.722601 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.722575 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:39.725872 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.725630 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7c6ng\"" Jul 15 12:31:39.725872 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.725654 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 12:31:39.730448 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.730389 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:39.774396 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.774362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glwkc\" (UniqueName: \"kubernetes.io/projected/40257c89-87c6-42bf-a333-09910d41f0c4-kube-api-access-glwkc\") pod \"limitador-limitador-64c94fd64f-p4jdj\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:39.774564 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.774541 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/40257c89-87c6-42bf-a333-09910d41f0c4-config-file\") pod \"limitador-limitador-64c94fd64f-p4jdj\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:39.815767 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.815722 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:39.875868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.875840 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glwkc\" (UniqueName: \"kubernetes.io/projected/40257c89-87c6-42bf-a333-09910d41f0c4-kube-api-access-glwkc\") pod \"limitador-limitador-64c94fd64f-p4jdj\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:39.876038 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.875967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/40257c89-87c6-42bf-a333-09910d41f0c4-config-file\") pod \"limitador-limitador-64c94fd64f-p4jdj\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:39.876693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.876666 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/40257c89-87c6-42bf-a333-09910d41f0c4-config-file\") pod \"limitador-limitador-64c94fd64f-p4jdj\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:39.885346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:39.885318 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glwkc\" (UniqueName: \"kubernetes.io/projected/40257c89-87c6-42bf-a333-09910d41f0c4-kube-api-access-glwkc\") pod \"limitador-limitador-64c94fd64f-p4jdj\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:40.037906 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.037821 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:40.450879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.450850 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:40.452127 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:40.452105 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40257c89_87c6_42bf_a333_09910d41f0c4.slice/crio-874748f3c09086f956ac5cb09f78bcffb8b1622c35124d4072092f6afa0f8b0e WatchSource:0}: Error finding container 874748f3c09086f956ac5cb09f78bcffb8b1622c35124d4072092f6afa0f8b0e: Status 404 returned error can't find the container with id 874748f3c09086f956ac5cb09f78bcffb8b1622c35124d4072092f6afa0f8b0e Jul 15 12:31:40.690981 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.690893 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qprbn" event={"ID":"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762","Type":"ContainerStarted","Data":"196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973"} Jul 15 12:31:40.692176 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.692145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" event={"ID":"40257c89-87c6-42bf-a333-09910d41f0c4","Type":"ContainerStarted","Data":"874748f3c09086f956ac5cb09f78bcffb8b1622c35124d4072092f6afa0f8b0e"} Jul 15 12:31:40.693523 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.693500 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2rtpn" event={"ID":"2b528544-111f-499a-8a00-1f951ab0f58d","Type":"ContainerStarted","Data":"cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037"} Jul 15 12:31:40.709479 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.709430 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qprbn" podStartSLOduration=1.485023361 podStartE2EDuration="4.70941691s" totalImagesPullingTime="3.224393549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:40.707371987 +0000 UTC m=+501.258131519" watchObservedRunningTime="2026-07-15 12:31:40.70941691 +0000 UTC m=+501.260176442" Jul 15 12:31:40.728602 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.728563 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-2rtpn" podStartSLOduration=1.382114359 podStartE2EDuration="4.728550576s" totalImagesPullingTime="3.346436217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:40.725160291 +0000 UTC m=+501.275919821" watchObservedRunningTime="2026-07-15 12:31:40.728550576 +0000 UTC m=+501.279310107" Jul 15 12:31:40.754189 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:40.754161 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2rtpn"] Jul 15 12:31:41.859673 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:41.859645 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:41.860145 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:41.859699 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:41.863963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:41.863944 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:42.708873 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:42.708830 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-2rtpn" podUID="2b528544-111f-499a-8a00-1f951ab0f58d" containerName="authorino" containerID="cri-o://cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037" gracePeriod=30 Jul 15 12:31:42.715857 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:42.715791 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-744ff8f984-2lmqx" Jul 15 12:31:42.791759 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:42.791723 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54975b4c7-f9qjr"] Jul 15 12:31:43.018341 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.018313 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:43.111871 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.111841 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qp9r\" (UniqueName: \"kubernetes.io/projected/2b528544-111f-499a-8a00-1f951ab0f58d-kube-api-access-9qp9r\") pod \"2b528544-111f-499a-8a00-1f951ab0f58d\" (UID: \"2b528544-111f-499a-8a00-1f951ab0f58d\") " Jul 15 12:31:43.114136 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.114105 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b528544-111f-499a-8a00-1f951ab0f58d-kube-api-access-9qp9r" (OuterVolumeSpecName: "kube-api-access-9qp9r") pod "2b528544-111f-499a-8a00-1f951ab0f58d" (UID: "2b528544-111f-499a-8a00-1f951ab0f58d"). InnerVolumeSpecName "kube-api-access-9qp9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:31:43.213517 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.213482 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qp9r\" (UniqueName: \"kubernetes.io/projected/2b528544-111f-499a-8a00-1f951ab0f58d-kube-api-access-9qp9r\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:43.717012 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.716977 2559 generic.go:356] "Generic (PLEG): container finished" podID="2b528544-111f-499a-8a00-1f951ab0f58d" containerID="cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037" exitCode=0 Jul 15 12:31:43.717195 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.717028 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2rtpn" Jul 15 12:31:43.717362 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.717326 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2rtpn" event={"ID":"2b528544-111f-499a-8a00-1f951ab0f58d","Type":"ContainerDied","Data":"cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037"} Jul 15 12:31:43.717484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.717379 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2rtpn" event={"ID":"2b528544-111f-499a-8a00-1f951ab0f58d","Type":"ContainerDied","Data":"0451eb53461f56d008e16fd01261accda8d973e90f1af40dc475681b099e7f56"} Jul 15 12:31:43.717484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.717400 2559 scope.go:117] "RemoveContainer" containerID="cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037" Jul 15 12:31:43.746960 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.746937 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2rtpn"] Jul 15 12:31:43.751425 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:43.751405 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-2rtpn"] Jul 15 12:31:44.111442 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:44.111365 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b528544-111f-499a-8a00-1f951ab0f58d" path="/var/lib/kubelet/pods/2b528544-111f-499a-8a00-1f951ab0f58d/volumes" Jul 15 12:31:44.179750 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:44.179727 2559 scope.go:117] "RemoveContainer" containerID="cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037" Jul 15 12:31:44.180124 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:44.180100 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037\": container with ID starting with cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037 not found: ID does not exist" containerID="cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037" Jul 15 12:31:44.180230 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:44.180135 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037"} err="failed to get container status \"cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037\": rpc error: code = NotFound desc = could not find container \"cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037\": container with ID starting with cff985bd639610c71440d5604a9ad1a69ac26ff334b224a221b98ca5fcc7e037 not found: ID does not exist" Jul 15 12:31:45.729839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:45.729791 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" event={"ID":"40257c89-87c6-42bf-a333-09910d41f0c4","Type":"ContainerStarted","Data":"111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7"} Jul 15 12:31:45.730211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:45.729934 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:45.755364 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:45.755307 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" podStartSLOduration=2.500278308 podStartE2EDuration="6.755291908s" totalImagesPullingTime="4.2550136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:45.754574251 +0000 UTC m=+506.305333792" watchObservedRunningTime="2026-07-15 12:31:45.755291908 +0000 UTC m=+506.306051440" Jul 15 12:31:54.921294 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:54.921246 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:54.922194 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:54.921451 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" podUID="40257c89-87c6-42bf-a333-09910d41f0c4" containerName="limitador" containerID="cri-o://111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7" gracePeriod=30 Jul 15 12:31:54.922576 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:54.922313 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:55.466919 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.466900 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:55.616963 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.616891 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/40257c89-87c6-42bf-a333-09910d41f0c4-config-file\") pod \"40257c89-87c6-42bf-a333-09910d41f0c4\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " Jul 15 12:31:55.617103 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.616974 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glwkc\" (UniqueName: \"kubernetes.io/projected/40257c89-87c6-42bf-a333-09910d41f0c4-kube-api-access-glwkc\") pod \"40257c89-87c6-42bf-a333-09910d41f0c4\" (UID: \"40257c89-87c6-42bf-a333-09910d41f0c4\") " Jul 15 12:31:55.617219 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.617188 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40257c89-87c6-42bf-a333-09910d41f0c4-config-file" (OuterVolumeSpecName: "config-file") pod "40257c89-87c6-42bf-a333-09910d41f0c4" (UID: "40257c89-87c6-42bf-a333-09910d41f0c4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:31:55.618929 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.618909 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40257c89-87c6-42bf-a333-09910d41f0c4-kube-api-access-glwkc" (OuterVolumeSpecName: "kube-api-access-glwkc") pod "40257c89-87c6-42bf-a333-09910d41f0c4" (UID: "40257c89-87c6-42bf-a333-09910d41f0c4"). InnerVolumeSpecName "kube-api-access-glwkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:31:55.718317 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.718293 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glwkc\" (UniqueName: \"kubernetes.io/projected/40257c89-87c6-42bf-a333-09910d41f0c4-kube-api-access-glwkc\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:55.718317 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.718316 2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/40257c89-87c6-42bf-a333-09910d41f0c4-config-file\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:31:55.782424 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.782399 2559 generic.go:356] "Generic (PLEG): container finished" podID="40257c89-87c6-42bf-a333-09910d41f0c4" containerID="111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7" exitCode=0 Jul 15 12:31:55.782549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.782442 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" Jul 15 12:31:55.782549 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.782505 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" event={"ID":"40257c89-87c6-42bf-a333-09910d41f0c4","Type":"ContainerDied","Data":"111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7"} Jul 15 12:31:55.782632 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.782552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-p4jdj" event={"ID":"40257c89-87c6-42bf-a333-09910d41f0c4","Type":"ContainerDied","Data":"874748f3c09086f956ac5cb09f78bcffb8b1622c35124d4072092f6afa0f8b0e"} Jul 15 12:31:55.782632 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.782576 2559 scope.go:117] "RemoveContainer" containerID="111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7" Jul 15 12:31:55.791854 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.791838 2559 scope.go:117] "RemoveContainer" containerID="111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7" Jul 15 12:31:55.792107 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:31:55.792090 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7\": container with ID starting with 111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7 not found: ID does not exist" containerID="111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7" Jul 15 12:31:55.792154 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.792115 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7"} err="failed to get container status \"111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7\": rpc error: code = NotFound desc = could not find container \"111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7\": container with ID starting with 111fb39f0673515ba2cddc67885903fe9ef6de4e0c940322c8c52dddef1301e7 not found: ID does not exist" Jul 15 12:31:55.813438 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.813396 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:55.815338 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:55.815316 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-p4jdj"] Jul 15 12:31:56.110329 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:56.110303 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40257c89-87c6-42bf-a333-09910d41f0c4" path="/var/lib/kubelet/pods/40257c89-87c6-42bf-a333-09910d41f0c4/volumes" Jul 15 12:31:57.642688 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.642652 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-b92t2"] Jul 15 12:31:57.643064 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.643049 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40257c89-87c6-42bf-a333-09910d41f0c4" containerName="limitador" Jul 15 12:31:57.643064 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.643065 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="40257c89-87c6-42bf-a333-09910d41f0c4" containerName="limitador" Jul 15 12:31:57.643139 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.643081 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b528544-111f-499a-8a00-1f951ab0f58d" containerName="authorino" Jul 15 12:31:57.643139 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.643087 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b528544-111f-499a-8a00-1f951ab0f58d" containerName="authorino" Jul 15 12:31:57.643199 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.643148 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b528544-111f-499a-8a00-1f951ab0f58d" containerName="authorino" Jul 15 12:31:57.643199 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.643160 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="40257c89-87c6-42bf-a333-09910d41f0c4" containerName="limitador" Jul 15 12:31:57.647654 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.647636 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.650971 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.650952 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-svp7d\"" Jul 15 12:31:57.651202 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.651183 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 12:31:57.651921 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.651908 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 15 12:31:57.651980 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.651947 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:31:57.656700 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.656675 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-b92t2"] Jul 15 12:31:57.735430 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.735402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p62tz\" (UniqueName: \"kubernetes.io/projected/a3faad9f-532f-4943-95be-1a5049cc9024-kube-api-access-p62tz\") pod \"postgres-69fb4594cf-b92t2\" (UID: \"a3faad9f-532f-4943-95be-1a5049cc9024\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.735541 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.735455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a3faad9f-532f-4943-95be-1a5049cc9024-data\") pod \"postgres-69fb4594cf-b92t2\" (UID: \"a3faad9f-532f-4943-95be-1a5049cc9024\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.836071 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.836046 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p62tz\" (UniqueName: \"kubernetes.io/projected/a3faad9f-532f-4943-95be-1a5049cc9024-kube-api-access-p62tz\") pod \"postgres-69fb4594cf-b92t2\" (UID: \"a3faad9f-532f-4943-95be-1a5049cc9024\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.836197 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.836092 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a3faad9f-532f-4943-95be-1a5049cc9024-data\") pod \"postgres-69fb4594cf-b92t2\" (UID: \"a3faad9f-532f-4943-95be-1a5049cc9024\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.836511 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.836492 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a3faad9f-532f-4943-95be-1a5049cc9024-data\") pod \"postgres-69fb4594cf-b92t2\" (UID: \"a3faad9f-532f-4943-95be-1a5049cc9024\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.847964 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.847943 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p62tz\" (UniqueName: \"kubernetes.io/projected/a3faad9f-532f-4943-95be-1a5049cc9024-kube-api-access-p62tz\") pod \"postgres-69fb4594cf-b92t2\" (UID: \"a3faad9f-532f-4943-95be-1a5049cc9024\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:57.958013 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:57.957960 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:31:58.088599 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:58.088572 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-b92t2"] Jul 15 12:31:58.089520 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:31:58.089500 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3faad9f_532f_4943_95be_1a5049cc9024.slice/crio-749b6b03907c57cd916ed423558ba21b24a5f389c11b719ddc951994e2fae8c8 WatchSource:0}: Error finding container 749b6b03907c57cd916ed423558ba21b24a5f389c11b719ddc951994e2fae8c8: Status 404 returned error can't find the container with id 749b6b03907c57cd916ed423558ba21b24a5f389c11b719ddc951994e2fae8c8 Jul 15 12:31:58.800123 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:31:58.800089 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" event={"ID":"a3faad9f-532f-4943-95be-1a5049cc9024","Type":"ContainerStarted","Data":"749b6b03907c57cd916ed423558ba21b24a5f389c11b719ddc951994e2fae8c8"} Jul 15 12:32:05.842694 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:05.842654 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" event={"ID":"a3faad9f-532f-4943-95be-1a5049cc9024","Type":"ContainerStarted","Data":"531e4f6f784d7bc91fa1cb3cff595285f8a32653f2759e7fa91a7051631f04d2"} Jul 15 12:32:05.843072 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:05.842730 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:32:05.864625 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:05.864578 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" podStartSLOduration=2.109733699 podStartE2EDuration="8.864563829s" totalImagesPullingTime="6.75483013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:32:05.862173737 +0000 UTC m=+526.412933297" watchObservedRunningTime="2026-07-15 12:32:05.864563829 +0000 UTC m=+526.415323420" Jul 15 12:32:07.830462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:07.830392 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54975b4c7-f9qjr" podUID="9572266e-06ef-4f42-b3e3-205953ed2ac7" containerName="console" containerID="cri-o://1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5" gracePeriod=15 Jul 15 12:32:08.082782 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.082728 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:32:08.227716 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227686 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-trusted-ca-bundle\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.227893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227736 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-service-ca\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.227893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227763 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk57b\" (UniqueName: \"kubernetes.io/projected/9572266e-06ef-4f42-b3e3-205953ed2ac7-kube-api-access-nk57b\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.227893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227849 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-oauth-serving-cert\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.227893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227879 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-config\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.228115 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227926 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-oauth-config\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.228115 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.227957 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-serving-cert\") pod \"9572266e-06ef-4f42-b3e3-205953ed2ac7\" (UID: \"9572266e-06ef-4f42-b3e3-205953ed2ac7\") " Jul 15 12:32:08.228247 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.228216 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:32:08.228320 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.228246 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-service-ca" (OuterVolumeSpecName: "service-ca") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:32:08.228320 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.228278 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:32:08.228320 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.228300 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-config" (OuterVolumeSpecName: "console-config") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:32:08.230024 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.230003 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:32:08.230116 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.230030 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:32:08.230153 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.230121 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9572266e-06ef-4f42-b3e3-205953ed2ac7-kube-api-access-nk57b" (OuterVolumeSpecName: "kube-api-access-nk57b") pod "9572266e-06ef-4f42-b3e3-205953ed2ac7" (UID: "9572266e-06ef-4f42-b3e3-205953ed2ac7"). InnerVolumeSpecName "kube-api-access-nk57b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:32:08.329355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329331 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-config\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.329355 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329352 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-oauth-config\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.329507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329361 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9572266e-06ef-4f42-b3e3-205953ed2ac7-console-serving-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.329507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329370 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-trusted-ca-bundle\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.329507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329378 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.329507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329396 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk57b\" (UniqueName: \"kubernetes.io/projected/9572266e-06ef-4f42-b3e3-205953ed2ac7-kube-api-access-nk57b\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.329507 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.329404 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9572266e-06ef-4f42-b3e3-205953ed2ac7-oauth-serving-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:08.860497 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.860469 2559 generic.go:356] "Generic (PLEG): container finished" podID="9572266e-06ef-4f42-b3e3-205953ed2ac7" containerID="1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5" exitCode=0 Jul 15 12:32:08.860868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.860505 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54975b4c7-f9qjr" event={"ID":"9572266e-06ef-4f42-b3e3-205953ed2ac7","Type":"ContainerDied","Data":"1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5"} Jul 15 12:32:08.860868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.860507 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54975b4c7-f9qjr" Jul 15 12:32:08.860868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.860549 2559 scope.go:117] "RemoveContainer" containerID="1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5" Jul 15 12:32:08.860868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.860538 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54975b4c7-f9qjr" event={"ID":"9572266e-06ef-4f42-b3e3-205953ed2ac7","Type":"ContainerDied","Data":"664be6f39fdfd11383f6feccc2947d2b241a954006d7fffc6f094f160bb6e623"} Jul 15 12:32:08.870096 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.870074 2559 scope.go:117] "RemoveContainer" containerID="1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5" Jul 15 12:32:08.870363 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:32:08.870344 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5\": container with ID starting with 1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5 not found: ID does not exist" containerID="1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5" Jul 15 12:32:08.870415 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.870371 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5"} err="failed to get container status \"1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5\": rpc error: code = NotFound desc = could not find container \"1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5\": container with ID starting with 1b62c2c94f7ea319ae549abcb50d12d44c0edecfc7543d39d886cd9093f18dc5 not found: ID does not exist" Jul 15 12:32:08.883400 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.883381 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54975b4c7-f9qjr"] Jul 15 12:32:08.885585 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:08.885566 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54975b4c7-f9qjr"] Jul 15 12:32:10.111829 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:10.111778 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9572266e-06ef-4f42-b3e3-205953ed2ac7" path="/var/lib/kubelet/pods/9572266e-06ef-4f42-b3e3-205953ed2ac7/volumes" Jul 15 12:32:11.877363 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:11.877337 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-b92t2" Jul 15 12:32:12.509375 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.509348 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pq2vl"] Jul 15 12:32:12.509930 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.509900 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9572266e-06ef-4f42-b3e3-205953ed2ac7" containerName="console" Jul 15 12:32:12.509930 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.509924 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9572266e-06ef-4f42-b3e3-205953ed2ac7" containerName="console" Jul 15 12:32:12.510096 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.510012 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9572266e-06ef-4f42-b3e3-205953ed2ac7" containerName="console" Jul 15 12:32:12.534664 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.534641 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pq2vl"] Jul 15 12:32:12.534815 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.534719 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:12.663267 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.663242 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42pnf\" (UniqueName: \"kubernetes.io/projected/53298653-af8d-4202-89f0-072fcc111bf2-kube-api-access-42pnf\") pod \"authorino-5d4b67c446-pq2vl\" (UID: \"53298653-af8d-4202-89f0-072fcc111bf2\") " pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:12.706716 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.706690 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pq2vl"] Jul 15 12:32:12.706942 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:32:12.706921 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-42pnf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-pq2vl" podUID="53298653-af8d-4202-89f0-072fcc111bf2" Jul 15 12:32:12.744034 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.744012 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-5j6lk"] Jul 15 12:32:12.747151 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.747137 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.750037 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.750021 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 12:32:12.756566 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.756544 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-5j6lk"] Jul 15 12:32:12.764292 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.764248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42pnf\" (UniqueName: \"kubernetes.io/projected/53298653-af8d-4202-89f0-072fcc111bf2-kube-api-access-42pnf\") pod \"authorino-5d4b67c446-pq2vl\" (UID: \"53298653-af8d-4202-89f0-072fcc111bf2\") " pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:12.782257 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.782236 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-5j6lk"] Jul 15 12:32:12.782636 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:32:12.782611 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-q8mzt tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-q8mzt tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-5j6lk" podUID="f90daf88-3ee5-4fbc-8415-a5804777cbb4" Jul 15 12:32:12.786576 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.786556 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42pnf\" (UniqueName: \"kubernetes.io/projected/53298653-af8d-4202-89f0-072fcc111bf2-kube-api-access-42pnf\") pod \"authorino-5d4b67c446-pq2vl\" (UID: \"53298653-af8d-4202-89f0-072fcc111bf2\") " pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:12.817155 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.817135 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54cc9685df-pnv8f"] Jul 15 12:32:12.820142 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.820129 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:12.834978 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.834960 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54cc9685df-pnv8f"] Jul 15 12:32:12.865559 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.865536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8mzt\" (UniqueName: \"kubernetes.io/projected/f90daf88-3ee5-4fbc-8415-a5804777cbb4-kube-api-access-q8mzt\") pod \"authorino-76877994-5j6lk\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.865647 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.865573 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f90daf88-3ee5-4fbc-8415-a5804777cbb4-tls-cert\") pod \"authorino-76877994-5j6lk\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.883216 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.883196 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:12.883511 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.883202 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.889113 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.889096 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:12.892292 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.892275 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.966388 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.966365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8mzt\" (UniqueName: \"kubernetes.io/projected/f90daf88-3ee5-4fbc-8415-a5804777cbb4-kube-api-access-q8mzt\") pod \"authorino-76877994-5j6lk\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.966491 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.966404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzw8m\" (UniqueName: \"kubernetes.io/projected/ec262afd-0465-420e-a94d-d768905bf5e3-kube-api-access-lzw8m\") pod \"authorino-54cc9685df-pnv8f\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:12.966491 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.966444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f90daf88-3ee5-4fbc-8415-a5804777cbb4-tls-cert\") pod \"authorino-76877994-5j6lk\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.966572 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.966517 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec262afd-0465-420e-a94d-d768905bf5e3-tls-cert\") pod \"authorino-54cc9685df-pnv8f\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:12.968757 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.968739 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f90daf88-3ee5-4fbc-8415-a5804777cbb4-tls-cert\") pod \"authorino-76877994-5j6lk\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:12.976169 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:12.976152 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8mzt\" (UniqueName: \"kubernetes.io/projected/f90daf88-3ee5-4fbc-8415-a5804777cbb4-kube-api-access-q8mzt\") pod \"authorino-76877994-5j6lk\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:13.067528 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.067478 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8mzt\" (UniqueName: \"kubernetes.io/projected/f90daf88-3ee5-4fbc-8415-a5804777cbb4-kube-api-access-q8mzt\") pod \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " Jul 15 12:32:13.067528 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.067510 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42pnf\" (UniqueName: \"kubernetes.io/projected/53298653-af8d-4202-89f0-072fcc111bf2-kube-api-access-42pnf\") pod \"53298653-af8d-4202-89f0-072fcc111bf2\" (UID: \"53298653-af8d-4202-89f0-072fcc111bf2\") " Jul 15 12:32:13.067629 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.067542 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f90daf88-3ee5-4fbc-8415-a5804777cbb4-tls-cert\") pod \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\" (UID: \"f90daf88-3ee5-4fbc-8415-a5804777cbb4\") " Jul 15 12:32:13.067738 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.067710 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec262afd-0465-420e-a94d-d768905bf5e3-tls-cert\") pod \"authorino-54cc9685df-pnv8f\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:13.067879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.067854 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzw8m\" (UniqueName: \"kubernetes.io/projected/ec262afd-0465-420e-a94d-d768905bf5e3-kube-api-access-lzw8m\") pod \"authorino-54cc9685df-pnv8f\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:13.069587 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.069566 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53298653-af8d-4202-89f0-072fcc111bf2-kube-api-access-42pnf" (OuterVolumeSpecName: "kube-api-access-42pnf") pod "53298653-af8d-4202-89f0-072fcc111bf2" (UID: "53298653-af8d-4202-89f0-072fcc111bf2"). InnerVolumeSpecName "kube-api-access-42pnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:32:13.069678 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.069628 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90daf88-3ee5-4fbc-8415-a5804777cbb4-kube-api-access-q8mzt" (OuterVolumeSpecName: "kube-api-access-q8mzt") pod "f90daf88-3ee5-4fbc-8415-a5804777cbb4" (UID: "f90daf88-3ee5-4fbc-8415-a5804777cbb4"). InnerVolumeSpecName "kube-api-access-q8mzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:32:13.069792 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.069768 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f90daf88-3ee5-4fbc-8415-a5804777cbb4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f90daf88-3ee5-4fbc-8415-a5804777cbb4" (UID: "f90daf88-3ee5-4fbc-8415-a5804777cbb4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:32:13.070164 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.070146 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec262afd-0465-420e-a94d-d768905bf5e3-tls-cert\") pod \"authorino-54cc9685df-pnv8f\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:13.078238 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.078218 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzw8m\" (UniqueName: \"kubernetes.io/projected/ec262afd-0465-420e-a94d-d768905bf5e3-kube-api-access-lzw8m\") pod \"authorino-54cc9685df-pnv8f\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:13.129026 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.129006 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:32:13.168671 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.168646 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8mzt\" (UniqueName: \"kubernetes.io/projected/f90daf88-3ee5-4fbc-8415-a5804777cbb4-kube-api-access-q8mzt\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:13.168671 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.168668 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42pnf\" (UniqueName: \"kubernetes.io/projected/53298653-af8d-4202-89f0-072fcc111bf2-kube-api-access-42pnf\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:13.168841 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.168679 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f90daf88-3ee5-4fbc-8415-a5804777cbb4-tls-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:13.294752 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.294729 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54cc9685df-pnv8f"] Jul 15 12:32:13.295427 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:32:13.295405 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec262afd_0465_420e_a94d_d768905bf5e3.slice/crio-48759c0178a1a80b228a4800cb5f38ccf3c228d68855716465c7cc503a517af9 WatchSource:0}: Error finding container 48759c0178a1a80b228a4800cb5f38ccf3c228d68855716465c7cc503a517af9: Status 404 returned error can't find the container with id 48759c0178a1a80b228a4800cb5f38ccf3c228d68855716465c7cc503a517af9 Jul 15 12:32:13.889525 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.889499 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5j6lk" Jul 15 12:32:13.889936 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.889526 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pq2vl" Jul 15 12:32:13.889936 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.889500 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cc9685df-pnv8f" event={"ID":"ec262afd-0465-420e-a94d-d768905bf5e3","Type":"ContainerStarted","Data":"48759c0178a1a80b228a4800cb5f38ccf3c228d68855716465c7cc503a517af9"} Jul 15 12:32:13.924877 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.924846 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-5j6lk"] Jul 15 12:32:13.929145 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.929125 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-5j6lk"] Jul 15 12:32:13.952126 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.952099 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pq2vl"] Jul 15 12:32:13.956307 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:13.956287 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pq2vl"] Jul 15 12:32:14.110247 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:14.110227 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53298653-af8d-4202-89f0-072fcc111bf2" path="/var/lib/kubelet/pods/53298653-af8d-4202-89f0-072fcc111bf2/volumes" Jul 15 12:32:14.110485 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:14.110473 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f90daf88-3ee5-4fbc-8415-a5804777cbb4" path="/var/lib/kubelet/pods/f90daf88-3ee5-4fbc-8415-a5804777cbb4/volumes" Jul 15 12:32:14.900773 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:14.900742 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cc9685df-pnv8f" event={"ID":"ec262afd-0465-420e-a94d-d768905bf5e3","Type":"ContainerStarted","Data":"13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f"} Jul 15 12:32:14.919415 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:14.919374 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54cc9685df-pnv8f" podStartSLOduration=1.9424458100000002 podStartE2EDuration="2.91936068s" totalImagesPullingTime="976.91487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:32:14.918124008 +0000 UTC m=+535.468883561" watchObservedRunningTime="2026-07-15 12:32:14.91936068 +0000 UTC m=+535.470120210" Jul 15 12:32:14.948509 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:14.948474 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qprbn"] Jul 15 12:32:14.948656 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:14.948616 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qprbn" podUID="d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" containerName="authorino" containerID="cri-o://196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973" gracePeriod=30 Jul 15 12:32:15.195294 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.195272 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:32:15.289384 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.289358 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762-kube-api-access-p2mzq\") pod \"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762\" (UID: \"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762\") " Jul 15 12:32:15.291385 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.291363 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762-kube-api-access-p2mzq" (OuterVolumeSpecName: "kube-api-access-p2mzq") pod "d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" (UID: "d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762"). InnerVolumeSpecName "kube-api-access-p2mzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:32:15.390535 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.390515 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762-kube-api-access-p2mzq\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:15.907748 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.907713 2559 generic.go:356] "Generic (PLEG): container finished" podID="d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" containerID="196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973" exitCode=0 Jul 15 12:32:15.907748 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.907752 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qprbn" Jul 15 12:32:15.908316 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.907787 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qprbn" event={"ID":"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762","Type":"ContainerDied","Data":"196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973"} Jul 15 12:32:15.908316 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.907846 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qprbn" event={"ID":"d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762","Type":"ContainerDied","Data":"6c1cd7ad2e7657108c7df7f5b39d3b6e72746fef96a4da2997078b9c831512e9"} Jul 15 12:32:15.908316 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.907863 2559 scope.go:117] "RemoveContainer" containerID="196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973" Jul 15 12:32:15.917138 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.917125 2559 scope.go:117] "RemoveContainer" containerID="196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973" Jul 15 12:32:15.917383 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:32:15.917364 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973\": container with ID starting with 196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973 not found: ID does not exist" containerID="196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973" Jul 15 12:32:15.917437 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.917389 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973"} err="failed to get container status \"196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973\": rpc error: code = NotFound desc = could not find container \"196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973\": container with ID starting with 196909598b46b4375174259d741635c084337564aa11ec3225f96d5f0ad18973 not found: ID does not exist" Jul 15 12:32:15.938860 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.938834 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qprbn"] Jul 15 12:32:15.950617 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:15.950597 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qprbn"] Jul 15 12:32:16.109571 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:16.109549 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" path="/var/lib/kubelet/pods/d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762/volumes" Jul 15 12:32:20.336936 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.336861 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f79677757-zxts7"] Jul 15 12:32:20.337303 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.337287 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" containerName="authorino" Jul 15 12:32:20.337346 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.337305 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" containerName="authorino" Jul 15 12:32:20.337383 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.337377 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6bfc0cd-74bf-4856-bc0e-9e8f8a61e762" containerName="authorino" Jul 15 12:32:20.347393 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.347373 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.353016 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.352996 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6rwqv\"" Jul 15 12:32:20.353166 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.353147 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 12:32:20.355064 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.355047 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 12:32:20.385838 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.385815 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f79677757-zxts7"] Jul 15 12:32:20.510988 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.510961 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fc57dd599-28x87"] Jul 15 12:32:20.514720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.514705 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.529003 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.528982 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-28x87"] Jul 15 12:32:20.533724 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.533706 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9043e132-10f6-46ed-9df1-709b330d6093-webhook-cert\") pod \"maas-controller-6f79677757-zxts7\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.533820 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.533741 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmxvz\" (UniqueName: \"kubernetes.io/projected/9043e132-10f6-46ed-9df1-709b330d6093-kube-api-access-wmxvz\") pod \"maas-controller-6f79677757-zxts7\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.634456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.634403 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gj6z\" (UniqueName: \"kubernetes.io/projected/0af794a5-6517-434e-b4d2-b9b51e0861b5-kube-api-access-5gj6z\") pod \"maas-controller-7fc57dd599-28x87\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.634456 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.634451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0af794a5-6517-434e-b4d2-b9b51e0861b5-webhook-cert\") pod \"maas-controller-7fc57dd599-28x87\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.634573 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.634488 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9043e132-10f6-46ed-9df1-709b330d6093-webhook-cert\") pod \"maas-controller-6f79677757-zxts7\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.634573 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.634517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmxvz\" (UniqueName: \"kubernetes.io/projected/9043e132-10f6-46ed-9df1-709b330d6093-kube-api-access-wmxvz\") pod \"maas-controller-6f79677757-zxts7\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.636864 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.636836 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9043e132-10f6-46ed-9df1-709b330d6093-webhook-cert\") pod \"maas-controller-6f79677757-zxts7\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.647212 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.647194 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmxvz\" (UniqueName: \"kubernetes.io/projected/9043e132-10f6-46ed-9df1-709b330d6093-kube-api-access-wmxvz\") pod \"maas-controller-6f79677757-zxts7\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.657032 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.657014 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:20.735595 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.735566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gj6z\" (UniqueName: \"kubernetes.io/projected/0af794a5-6517-434e-b4d2-b9b51e0861b5-kube-api-access-5gj6z\") pod \"maas-controller-7fc57dd599-28x87\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.735738 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.735651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0af794a5-6517-434e-b4d2-b9b51e0861b5-webhook-cert\") pod \"maas-controller-7fc57dd599-28x87\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.739463 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.739438 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0af794a5-6517-434e-b4d2-b9b51e0861b5-webhook-cert\") pod \"maas-controller-7fc57dd599-28x87\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.745003 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.744979 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gj6z\" (UniqueName: \"kubernetes.io/projected/0af794a5-6517-434e-b4d2-b9b51e0861b5-kube-api-access-5gj6z\") pod \"maas-controller-7fc57dd599-28x87\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.824285 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.824257 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f79677757-zxts7"] Jul 15 12:32:20.824395 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.824353 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:20.825949 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:32:20.825926 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9043e132_10f6_46ed_9df1_709b330d6093.slice/crio-9fe0c1418a446d9fd28e0e1bc83228ba4240d8b7c5843e0b43587dca7d57c765 WatchSource:0}: Error finding container 9fe0c1418a446d9fd28e0e1bc83228ba4240d8b7c5843e0b43587dca7d57c765: Status 404 returned error can't find the container with id 9fe0c1418a446d9fd28e0e1bc83228ba4240d8b7c5843e0b43587dca7d57c765 Jul 15 12:32:20.938403 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.938377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f79677757-zxts7" event={"ID":"9043e132-10f6-46ed-9df1-709b330d6093","Type":"ContainerStarted","Data":"9fe0c1418a446d9fd28e0e1bc83228ba4240d8b7c5843e0b43587dca7d57c765"} Jul 15 12:32:20.970809 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:20.970774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-28x87"] Jul 15 12:32:20.971796 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:32:20.971777 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af794a5_6517_434e_b4d2_b9b51e0861b5.slice/crio-6e81ea4bd7be3a90010e263f5c6ecdef1a3f9982d49be5ed474ed90c80d4fbdd WatchSource:0}: Error finding container 6e81ea4bd7be3a90010e263f5c6ecdef1a3f9982d49be5ed474ed90c80d4fbdd: Status 404 returned error can't find the container with id 6e81ea4bd7be3a90010e263f5c6ecdef1a3f9982d49be5ed474ed90c80d4fbdd Jul 15 12:32:21.949616 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:21.949579 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc57dd599-28x87" event={"ID":"0af794a5-6517-434e-b4d2-b9b51e0861b5","Type":"ContainerStarted","Data":"6e81ea4bd7be3a90010e263f5c6ecdef1a3f9982d49be5ed474ed90c80d4fbdd"} Jul 15 12:32:23.964181 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:23.964149 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f79677757-zxts7" event={"ID":"9043e132-10f6-46ed-9df1-709b330d6093","Type":"ContainerStarted","Data":"edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95"} Jul 15 12:32:23.964659 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:23.964260 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:23.966063 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:23.966034 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc57dd599-28x87" event={"ID":"0af794a5-6517-434e-b4d2-b9b51e0861b5","Type":"ContainerStarted","Data":"16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50"} Jul 15 12:32:23.966174 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:23.966105 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:23.995335 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:23.995274 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f79677757-zxts7" podStartSLOduration=1.308696626 podStartE2EDuration="3.995257431s" totalImagesPullingTime="2.686560805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:32:23.993680181 +0000 UTC m=+544.544439713" watchObservedRunningTime="2026-07-15 12:32:23.995257431 +0000 UTC m=+544.546016963" Jul 15 12:32:24.022541 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:24.022496 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fc57dd599-28x87" podStartSLOduration=1.470369998 podStartE2EDuration="4.022482705s" totalImagesPullingTime="2.552112707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:32:24.022386924 +0000 UTC m=+544.573146489" watchObservedRunningTime="2026-07-15 12:32:24.022482705 +0000 UTC m=+544.573242235" Jul 15 12:32:34.974159 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:34.974124 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:32:34.974712 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:34.974225 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:35.056821 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.056780 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f79677757-zxts7"] Jul 15 12:32:35.057027 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.057006 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f79677757-zxts7" podUID="9043e132-10f6-46ed-9df1-709b330d6093" containerName="manager" containerID="cri-o://edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95" gracePeriod=30 Jul 15 12:32:35.848690 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.848662 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:35.942927 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.942905 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9043e132-10f6-46ed-9df1-709b330d6093-webhook-cert\") pod \"9043e132-10f6-46ed-9df1-709b330d6093\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " Jul 15 12:32:35.943071 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.942954 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmxvz\" (UniqueName: \"kubernetes.io/projected/9043e132-10f6-46ed-9df1-709b330d6093-kube-api-access-wmxvz\") pod \"9043e132-10f6-46ed-9df1-709b330d6093\" (UID: \"9043e132-10f6-46ed-9df1-709b330d6093\") " Jul 15 12:32:35.945027 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.944999 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9043e132-10f6-46ed-9df1-709b330d6093-kube-api-access-wmxvz" (OuterVolumeSpecName: "kube-api-access-wmxvz") pod "9043e132-10f6-46ed-9df1-709b330d6093" (UID: "9043e132-10f6-46ed-9df1-709b330d6093"). InnerVolumeSpecName "kube-api-access-wmxvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:32:35.945124 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:35.945035 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9043e132-10f6-46ed-9df1-709b330d6093-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9043e132-10f6-46ed-9df1-709b330d6093" (UID: "9043e132-10f6-46ed-9df1-709b330d6093"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:32:36.033580 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.033551 2559 generic.go:356] "Generic (PLEG): container finished" podID="9043e132-10f6-46ed-9df1-709b330d6093" containerID="edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95" exitCode=0 Jul 15 12:32:36.033893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.033586 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f79677757-zxts7" event={"ID":"9043e132-10f6-46ed-9df1-709b330d6093","Type":"ContainerDied","Data":"edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95"} Jul 15 12:32:36.033893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.033593 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f79677757-zxts7" Jul 15 12:32:36.033893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.033627 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f79677757-zxts7" event={"ID":"9043e132-10f6-46ed-9df1-709b330d6093","Type":"ContainerDied","Data":"9fe0c1418a446d9fd28e0e1bc83228ba4240d8b7c5843e0b43587dca7d57c765"} Jul 15 12:32:36.033893 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.033650 2559 scope.go:117] "RemoveContainer" containerID="edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95" Jul 15 12:32:36.042551 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.042538 2559 scope.go:117] "RemoveContainer" containerID="edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95" Jul 15 12:32:36.042821 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:32:36.042787 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95\": container with ID starting with edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95 not found: ID does not exist" containerID="edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95" Jul 15 12:32:36.042863 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.042824 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95"} err="failed to get container status \"edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95\": rpc error: code = NotFound desc = could not find container \"edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95\": container with ID starting with edf6b1aae15e3820afeb1ebb8faef5eea957883a4a3b2065347a9a2cacac6a95 not found: ID does not exist" Jul 15 12:32:36.044128 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.044112 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9043e132-10f6-46ed-9df1-709b330d6093-webhook-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:36.044172 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.044132 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmxvz\" (UniqueName: \"kubernetes.io/projected/9043e132-10f6-46ed-9df1-709b330d6093-kube-api-access-wmxvz\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:32:36.057127 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.057108 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f79677757-zxts7"] Jul 15 12:32:36.064626 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.064607 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f79677757-zxts7"] Jul 15 12:32:36.109949 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:36.109928 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9043e132-10f6-46ed-9df1-709b330d6093" path="/var/lib/kubelet/pods/9043e132-10f6-46ed-9df1-709b330d6093/volumes" Jul 15 12:32:57.780868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.780836 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-xnldf"] Jul 15 12:32:57.781255 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.781248 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9043e132-10f6-46ed-9df1-709b330d6093" containerName="manager" Jul 15 12:32:57.781318 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.781278 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9043e132-10f6-46ed-9df1-709b330d6093" containerName="manager" Jul 15 12:32:57.781352 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.781348 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9043e132-10f6-46ed-9df1-709b330d6093" containerName="manager" Jul 15 12:32:57.787260 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.787239 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:57.797295 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.797259 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 12:32:57.814608 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.814582 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-xnldf"] Jul 15 12:32:57.816758 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.816734 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fmfd5"] Jul 15 12:32:57.819949 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.819927 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:57.823381 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.823362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5jl8q\"" Jul 15 12:32:57.852611 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.852580 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fmfd5"] Jul 15 12:32:57.926555 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.926519 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9nm\" (UniqueName: \"kubernetes.io/projected/1b0dabbe-e728-4305-89c1-6425a32e3918-kube-api-access-kb9nm\") pod \"payload-pre-processing-74584bb9f8-xnldf\" (UID: \"1b0dabbe-e728-4305-89c1-6425a32e3918\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:57.926733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.926634 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/783c2fa1-4ace-4f75-b866-f2f37de9fee9-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fmfd5\" (UID: \"783c2fa1-4ace-4f75-b866-f2f37de9fee9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:57.926733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.926663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vdxk\" (UniqueName: \"kubernetes.io/projected/783c2fa1-4ace-4f75-b866-f2f37de9fee9-kube-api-access-5vdxk\") pod \"payload-processing-f9b8d9b9b-fmfd5\" (UID: \"783c2fa1-4ace-4f75-b866-f2f37de9fee9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:57.926733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:57.926708 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b0dabbe-e728-4305-89c1-6425a32e3918-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-xnldf\" (UID: \"1b0dabbe-e728-4305-89c1-6425a32e3918\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:58.027160 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.027122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b0dabbe-e728-4305-89c1-6425a32e3918-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-xnldf\" (UID: \"1b0dabbe-e728-4305-89c1-6425a32e3918\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:58.027354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.027184 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9nm\" (UniqueName: \"kubernetes.io/projected/1b0dabbe-e728-4305-89c1-6425a32e3918-kube-api-access-kb9nm\") pod \"payload-pre-processing-74584bb9f8-xnldf\" (UID: \"1b0dabbe-e728-4305-89c1-6425a32e3918\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:58.027354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.027269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/783c2fa1-4ace-4f75-b866-f2f37de9fee9-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fmfd5\" (UID: \"783c2fa1-4ace-4f75-b866-f2f37de9fee9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:58.027354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.027297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vdxk\" (UniqueName: \"kubernetes.io/projected/783c2fa1-4ace-4f75-b866-f2f37de9fee9-kube-api-access-5vdxk\") pod \"payload-processing-f9b8d9b9b-fmfd5\" (UID: \"783c2fa1-4ace-4f75-b866-f2f37de9fee9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:58.027839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.027819 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b0dabbe-e728-4305-89c1-6425a32e3918-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-xnldf\" (UID: \"1b0dabbe-e728-4305-89c1-6425a32e3918\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:58.027991 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.027974 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/783c2fa1-4ace-4f75-b866-f2f37de9fee9-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fmfd5\" (UID: \"783c2fa1-4ace-4f75-b866-f2f37de9fee9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:58.055414 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.055334 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9nm\" (UniqueName: \"kubernetes.io/projected/1b0dabbe-e728-4305-89c1-6425a32e3918-kube-api-access-kb9nm\") pod \"payload-pre-processing-74584bb9f8-xnldf\" (UID: \"1b0dabbe-e728-4305-89c1-6425a32e3918\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:58.057060 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.057042 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vdxk\" (UniqueName: \"kubernetes.io/projected/783c2fa1-4ace-4f75-b866-f2f37de9fee9-kube-api-access-5vdxk\") pod \"payload-processing-f9b8d9b9b-fmfd5\" (UID: \"783c2fa1-4ace-4f75-b866-f2f37de9fee9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:58.097448 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.097409 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:32:58.130720 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.130683 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:32:58.294948 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.294920 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-xnldf"] Jul 15 12:32:58.300698 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:32:58.300671 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b0dabbe_e728_4305_89c1_6425a32e3918.slice/crio-8285807859b6447029a3450f51f0123cbefc3044f82a0b4720b0478209ad43ea WatchSource:0}: Error finding container 8285807859b6447029a3450f51f0123cbefc3044f82a0b4720b0478209ad43ea: Status 404 returned error can't find the container with id 8285807859b6447029a3450f51f0123cbefc3044f82a0b4720b0478209ad43ea Jul 15 12:32:58.347667 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:58.347643 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fmfd5"] Jul 15 12:32:58.350065 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:32:58.350035 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783c2fa1_4ace_4f75_b866_f2f37de9fee9.slice/crio-d0ca15a104dd1abc90adea1ab813c4119c4f0474849ad1da12cfa5ad6eb981ef WatchSource:0}: Error finding container d0ca15a104dd1abc90adea1ab813c4119c4f0474849ad1da12cfa5ad6eb981ef: Status 404 returned error can't find the container with id d0ca15a104dd1abc90adea1ab813c4119c4f0474849ad1da12cfa5ad6eb981ef Jul 15 12:32:59.191971 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:59.191928 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" event={"ID":"1b0dabbe-e728-4305-89c1-6425a32e3918","Type":"ContainerStarted","Data":"8285807859b6447029a3450f51f0123cbefc3044f82a0b4720b0478209ad43ea"} Jul 15 12:32:59.194630 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:32:59.194575 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" event={"ID":"783c2fa1-4ace-4f75-b866-f2f37de9fee9","Type":"ContainerStarted","Data":"d0ca15a104dd1abc90adea1ab813c4119c4f0474849ad1da12cfa5ad6eb981ef"} Jul 15 12:33:00.204098 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:00.204009 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" event={"ID":"783c2fa1-4ace-4f75-b866-f2f37de9fee9","Type":"ContainerStarted","Data":"eb25891868af2a0065a7b1e357c75e7f212f8c1c0fc1cf7c2da931d9988f70ab"} Jul 15 12:33:00.204098 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:00.204069 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:33:00.242911 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:00.242848 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" podStartSLOduration=1.67625287 podStartE2EDuration="3.242829919s" totalImagesPullingTime="1.566577049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:33:00.242510405 +0000 UTC m=+580.793269946" watchObservedRunningTime="2026-07-15 12:33:00.242829919 +0000 UTC m=+580.793589450" Jul 15 12:33:01.212095 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:01.212060 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" event={"ID":"1b0dabbe-e728-4305-89c1-6425a32e3918","Type":"ContainerStarted","Data":"51bf9d8a67b4b2c2c36d14bd789dd80803feb86d040e9fec00a29186e028b539"} Jul 15 12:33:01.212551 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:01.212131 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:33:01.239020 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:01.238974 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" podStartSLOduration=1.5826211909999999 podStartE2EDuration="4.238959743s" totalImagesPullingTime="2.656338552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:33:01.236734048 +0000 UTC m=+581.787493598" watchObservedRunningTime="2026-07-15 12:33:01.238959743 +0000 UTC m=+581.789719274" Jul 15 12:33:05.658846 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:05.658797 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j6q6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 12:33:05.659204 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:05.658868 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" podUID="2bc79424-ac1c-4dd0-9284-610be84542ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 12:33:06.252561 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:06.252528 2559 generic.go:356] "Generic (PLEG): container finished" podID="2bc79424-ac1c-4dd0-9284-610be84542ec" containerID="b982c10d02d70cdf3b1100cb6b1cdddad604cef3fabb64dd23c09795b829d55a" exitCode=0 Jul 15 12:33:06.252776 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:06.252599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" event={"ID":"2bc79424-ac1c-4dd0-9284-610be84542ec","Type":"ContainerDied","Data":"b982c10d02d70cdf3b1100cb6b1cdddad604cef3fabb64dd23c09795b829d55a"} Jul 15 12:33:06.252981 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:06.252968 2559 scope.go:117] "RemoveContainer" containerID="b982c10d02d70cdf3b1100cb6b1cdddad604cef3fabb64dd23c09795b829d55a" Jul 15 12:33:06.254002 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:06.253976 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:33:06.254064 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:06.254044 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:33:06.254099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:06.254082 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 12:33:07.260303 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:07.260270 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" event={"ID":"2bc79424-ac1c-4dd0-9284-610be84542ec","Type":"ContainerStarted","Data":"a9f2a63130adf1bcb55936103d91df0b81f3385f7ef70e7dbf4b9bda385e725a"} Jul 15 12:33:08.070348 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:08.070310 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:33:08.070568 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:08.070356 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j6q6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 12:33:08.070568 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:08.070428 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" podUID="2bc79424-ac1c-4dd0-9284-610be84542ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 12:33:09.071212 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:09.071175 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j6q6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 12:33:09.071848 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:09.071788 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" podUID="2bc79424-ac1c-4dd0-9284-610be84542ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 12:33:10.070616 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:10.070567 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j6q6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 12:33:10.070855 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:10.070640 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" podUID="2bc79424-ac1c-4dd0-9284-610be84542ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 12:33:11.074080 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:11.074050 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:33:11.074483 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:11.074337 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:33:11.075002 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:11.074984 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j6q6z" Jul 15 12:33:11.214178 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:11.214151 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-fmfd5" Jul 15 12:33:12.228327 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:12.228299 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-xnldf" Jul 15 12:33:19.940327 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:19.940295 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:33:19.940763 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:19.940384 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:33:33.744020 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.743986 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2"] Jul 15 12:33:33.750967 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.750944 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.755637 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.755612 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 12:33:33.755775 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.755647 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 12:33:33.755775 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.755663 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8v9kl\"" Jul 15 12:33:33.755775 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.755647 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 15 12:33:33.768695 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.768628 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2"] Jul 15 12:33:33.861081 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.861055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.861244 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.861095 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.861244 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.861114 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30115da7-3fe6-49f8-9dc6-8243232d9fa7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.861244 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.861162 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.861244 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.861185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvh58\" (UniqueName: \"kubernetes.io/projected/30115da7-3fe6-49f8-9dc6-8243232d9fa7-kube-api-access-vvh58\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.861404 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.861246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.962694 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.962663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.962694 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.962698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30115da7-3fe6-49f8-9dc6-8243232d9fa7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.962938 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.962744 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.962938 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.962767 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvh58\" (UniqueName: \"kubernetes.io/projected/30115da7-3fe6-49f8-9dc6-8243232d9fa7-kube-api-access-vvh58\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.962938 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.962860 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.963099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.962946 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.963243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.963209 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.963243 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.963233 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.963410 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.963382 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.965552 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.965524 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30115da7-3fe6-49f8-9dc6-8243232d9fa7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.965793 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.965777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30115da7-3fe6-49f8-9dc6-8243232d9fa7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:33.984056 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:33.984026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvh58\" (UniqueName: \"kubernetes.io/projected/30115da7-3fe6-49f8-9dc6-8243232d9fa7-kube-api-access-vvh58\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2\" (UID: \"30115da7-3fe6-49f8-9dc6-8243232d9fa7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:34.062262 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:34.062188 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:34.220191 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:34.220161 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2"] Jul 15 12:33:34.220472 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:33:34.220449 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30115da7_3fe6_49f8_9dc6_8243232d9fa7.slice/crio-f2ab7bfb449c0e347985a84d4825499c779df9f6f7ed0805ecd8123f53a82789 WatchSource:0}: Error finding container f2ab7bfb449c0e347985a84d4825499c779df9f6f7ed0805ecd8123f53a82789: Status 404 returned error can't find the container with id f2ab7bfb449c0e347985a84d4825499c779df9f6f7ed0805ecd8123f53a82789 Jul 15 12:33:34.221854 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:34.221836 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:33:34.433381 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:34.433289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" event={"ID":"30115da7-3fe6-49f8-9dc6-8243232d9fa7","Type":"ContainerStarted","Data":"f2ab7bfb449c0e347985a84d4825499c779df9f6f7ed0805ecd8123f53a82789"} Jul 15 12:33:36.446020 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:36.445982 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" event={"ID":"30115da7-3fe6-49f8-9dc6-8243232d9fa7","Type":"ContainerStarted","Data":"c02f0b9daa23096a34cfa94dd9258ca0224d83ebd83eaac9491022706cac6f05"} Jul 15 12:33:36.471095 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:36.471032 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" podStartSLOduration=2.258483217 podStartE2EDuration="3.471010752s" totalImagesPullingTime="1.212527535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:33:36.468577315 +0000 UTC m=+617.019336867" watchObservedRunningTime="2026-07-15 12:33:36.471010752 +0000 UTC m=+617.021770285" Jul 15 12:33:44.062674 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:44.062586 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:44.062674 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:44.062637 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:44.075127 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:44.075107 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:44.506227 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:44.506205 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2" Jul 15 12:33:50.418036 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.418006 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x"] Jul 15 12:33:50.422200 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.422178 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.424778 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.424758 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 12:33:50.430914 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.430891 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x"] Jul 15 12:33:50.501119 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.501097 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.501258 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.501214 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.501306 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.501253 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c92dac6-10a1-425b-bcb6-b93882073859-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.501306 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.501284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.501374 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.501307 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.501374 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.501359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4bf\" (UniqueName: \"kubernetes.io/projected/1c92dac6-10a1-425b-bcb6-b93882073859-kube-api-access-4n4bf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.602477 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.602450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.602477 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.602481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c92dac6-10a1-425b-bcb6-b93882073859-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.602693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.602498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.602693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.602512 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.602693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.602533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4bf\" (UniqueName: \"kubernetes.io/projected/1c92dac6-10a1-425b-bcb6-b93882073859-kube-api-access-4n4bf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.602693 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.602586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.603121 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.603006 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.603121 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.603041 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.603121 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.603057 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.604868 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.604847 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c92dac6-10a1-425b-bcb6-b93882073859-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.605272 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.605257 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c92dac6-10a1-425b-bcb6-b93882073859-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.612093 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.612073 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4bf\" (UniqueName: \"kubernetes.io/projected/1c92dac6-10a1-425b-bcb6-b93882073859-kube-api-access-4n4bf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48r7x\" (UID: \"1c92dac6-10a1-425b-bcb6-b93882073859\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.734772 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.734748 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:33:50.898772 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:50.898741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x"] Jul 15 12:33:50.899217 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:33:50.899186 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c92dac6_10a1_425b_bcb6_b93882073859.slice/crio-7a7c94b977d04a8341b22f17e44592b2901914adb38546b7c18f6910717d6b95 WatchSource:0}: Error finding container 7a7c94b977d04a8341b22f17e44592b2901914adb38546b7c18f6910717d6b95: Status 404 returned error can't find the container with id 7a7c94b977d04a8341b22f17e44592b2901914adb38546b7c18f6910717d6b95 Jul 15 12:33:51.538942 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:51.538902 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" event={"ID":"1c92dac6-10a1-425b-bcb6-b93882073859","Type":"ContainerStarted","Data":"7a7c94b977d04a8341b22f17e44592b2901914adb38546b7c18f6910717d6b95"} Jul 15 12:33:52.545703 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:52.545668 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" event={"ID":"1c92dac6-10a1-425b-bcb6-b93882073859","Type":"ContainerStarted","Data":"947a9fba8e0b66a487cde4c3407843422bbf90d18ced1c92827eced3d5fa2a3b"} Jul 15 12:33:52.568933 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:33:52.568888 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" podStartSLOduration=1.148570688 podStartE2EDuration="2.568872885s" totalImagesPullingTime="1.420302197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:33:52.566745629 +0000 UTC m=+633.117505165" watchObservedRunningTime="2026-07-15 12:33:52.568872885 +0000 UTC m=+633.119632416" Jul 15 12:34:00.734898 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:00.734867 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:34:00.735372 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:00.734958 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:34:00.747131 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:00.747108 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:34:01.608869 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:01.608835 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48r7x" Jul 15 12:34:35.842238 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.842192 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt"] Jul 15 12:34:35.845879 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.845860 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:35.848692 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.848671 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 15 12:34:35.857172 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.857150 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt"] Jul 15 12:34:35.983420 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.983389 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf8af06-b64a-4219-9d78-6b9398293541-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:35.983610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.983443 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:35.983610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.983494 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbfxd\" (UniqueName: \"kubernetes.io/projected/fcf8af06-b64a-4219-9d78-6b9398293541-kube-api-access-hbfxd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:35.983610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.983529 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:35.983610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.983593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:35.984129 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:35.983670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.084601 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.084571 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbfxd\" (UniqueName: \"kubernetes.io/projected/fcf8af06-b64a-4219-9d78-6b9398293541-kube-api-access-hbfxd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.084733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.084609 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.084733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.084650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.084733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.084703 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.084733 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.084722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf8af06-b64a-4219-9d78-6b9398293541-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.084957 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.084757 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.085188 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.085158 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.085288 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.085205 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.085288 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.085245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.086945 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.086916 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf8af06-b64a-4219-9d78-6b9398293541-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.087229 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.087212 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf8af06-b64a-4219-9d78-6b9398293541-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.094097 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.094047 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbfxd\" (UniqueName: \"kubernetes.io/projected/fcf8af06-b64a-4219-9d78-6b9398293541-kube-api-access-hbfxd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt\" (UID: \"fcf8af06-b64a-4219-9d78-6b9398293541\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.155731 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.155708 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:36.295110 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.295083 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt"] Jul 15 12:34:36.296577 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:34:36.296543 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcf8af06_b64a_4219_9d78_6b9398293541.slice/crio-f9448a0b908736c3ed6cf310f2b93178901efbfda62276a9ca5da67cc095f2fe WatchSource:0}: Error finding container f9448a0b908736c3ed6cf310f2b93178901efbfda62276a9ca5da67cc095f2fe: Status 404 returned error can't find the container with id f9448a0b908736c3ed6cf310f2b93178901efbfda62276a9ca5da67cc095f2fe Jul 15 12:34:36.798969 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.798933 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" event={"ID":"fcf8af06-b64a-4219-9d78-6b9398293541","Type":"ContainerStarted","Data":"c1e78a3e20e3683eac0332ec50344d494ae9bb29e37dfd4951ca71e7249bc521"} Jul 15 12:34:36.799111 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:36.798980 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" event={"ID":"fcf8af06-b64a-4219-9d78-6b9398293541","Type":"ContainerStarted","Data":"f9448a0b908736c3ed6cf310f2b93178901efbfda62276a9ca5da67cc095f2fe"} Jul 15 12:34:46.155852 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:46.155814 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:46.156430 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:46.155888 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:46.168298 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:46.168274 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:34:46.187897 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:46.187856 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" podStartSLOduration=10.789311087 podStartE2EDuration="11.187843524s" totalImagesPullingTime="398.532437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:34:36.824024769 +0000 UTC m=+677.374784300" watchObservedRunningTime="2026-07-15 12:34:46.187843524 +0000 UTC m=+686.738603052" Jul 15 12:34:46.869353 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:34:46.869322 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt" Jul 15 12:35:48.694898 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.694816 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c67f569c6-nftv7"] Jul 15 12:35:48.698367 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.698346 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:48.708289 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.708267 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c67f569c6-nftv7"] Jul 15 12:35:48.795642 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.795616 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gnv\" (UniqueName: \"kubernetes.io/projected/0f9af326-4b15-440d-9e94-d3aa668fbfa4-kube-api-access-p2gnv\") pod \"authorino-7c67f569c6-nftv7\" (UID: \"0f9af326-4b15-440d-9e94-d3aa668fbfa4\") " pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:48.795744 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.795669 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f9af326-4b15-440d-9e94-d3aa668fbfa4-tls-cert\") pod \"authorino-7c67f569c6-nftv7\" (UID: \"0f9af326-4b15-440d-9e94-d3aa668fbfa4\") " pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:48.896614 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.896590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gnv\" (UniqueName: \"kubernetes.io/projected/0f9af326-4b15-440d-9e94-d3aa668fbfa4-kube-api-access-p2gnv\") pod \"authorino-7c67f569c6-nftv7\" (UID: \"0f9af326-4b15-440d-9e94-d3aa668fbfa4\") " pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:48.896732 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.896658 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f9af326-4b15-440d-9e94-d3aa668fbfa4-tls-cert\") pod \"authorino-7c67f569c6-nftv7\" (UID: \"0f9af326-4b15-440d-9e94-d3aa668fbfa4\") " pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:48.899180 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.899153 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f9af326-4b15-440d-9e94-d3aa668fbfa4-tls-cert\") pod \"authorino-7c67f569c6-nftv7\" (UID: \"0f9af326-4b15-440d-9e94-d3aa668fbfa4\") " pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:48.906479 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:48.906456 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gnv\" (UniqueName: \"kubernetes.io/projected/0f9af326-4b15-440d-9e94-d3aa668fbfa4-kube-api-access-p2gnv\") pod \"authorino-7c67f569c6-nftv7\" (UID: \"0f9af326-4b15-440d-9e94-d3aa668fbfa4\") " pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:49.008072 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:49.008050 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c67f569c6-nftv7" Jul 15 12:35:49.156531 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:35:49.156506 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f9af326_4b15_440d_9e94_d3aa668fbfa4.slice/crio-9170052e19586f91b16523bd04aacc4fc83cbc4a29c0f81b8960bec973e1b727 WatchSource:0}: Error finding container 9170052e19586f91b16523bd04aacc4fc83cbc4a29c0f81b8960bec973e1b727: Status 404 returned error can't find the container with id 9170052e19586f91b16523bd04aacc4fc83cbc4a29c0f81b8960bec973e1b727 Jul 15 12:35:49.157640 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:49.157616 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c67f569c6-nftv7"] Jul 15 12:35:49.206658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:49.206630 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c67f569c6-nftv7" event={"ID":"0f9af326-4b15-440d-9e94-d3aa668fbfa4","Type":"ContainerStarted","Data":"9170052e19586f91b16523bd04aacc4fc83cbc4a29c0f81b8960bec973e1b727"} Jul 15 12:35:50.216205 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.216164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c67f569c6-nftv7" event={"ID":"0f9af326-4b15-440d-9e94-d3aa668fbfa4","Type":"ContainerStarted","Data":"1c05a1822c89ce0df06fbd821771fb66bea533912882d908d727377d8e22f1dc"} Jul 15 12:35:50.308453 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.308224 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c67f569c6-nftv7" podStartSLOduration=1.784203186 podStartE2EDuration="2.308201059s" totalImagesPullingTime="523.997873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:35:50.251766533 +0000 UTC m=+750.802526066" watchObservedRunningTime="2026-07-15 12:35:50.308201059 +0000 UTC m=+750.858960592" Jul 15 12:35:50.309466 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.309441 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54cc9685df-pnv8f"] Jul 15 12:35:50.309610 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.309597 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-54cc9685df-pnv8f" podUID="ec262afd-0465-420e-a94d-d768905bf5e3" containerName="authorino" containerID="cri-o://13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f" gracePeriod=30 Jul 15 12:35:50.575866 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.575840 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:35:50.712935 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.712901 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec262afd-0465-420e-a94d-d768905bf5e3-tls-cert\") pod \"ec262afd-0465-420e-a94d-d768905bf5e3\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " Jul 15 12:35:50.713103 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.713015 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzw8m\" (UniqueName: \"kubernetes.io/projected/ec262afd-0465-420e-a94d-d768905bf5e3-kube-api-access-lzw8m\") pod \"ec262afd-0465-420e-a94d-d768905bf5e3\" (UID: \"ec262afd-0465-420e-a94d-d768905bf5e3\") " Jul 15 12:35:50.715387 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.715359 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec262afd-0465-420e-a94d-d768905bf5e3-kube-api-access-lzw8m" (OuterVolumeSpecName: "kube-api-access-lzw8m") pod "ec262afd-0465-420e-a94d-d768905bf5e3" (UID: "ec262afd-0465-420e-a94d-d768905bf5e3"). InnerVolumeSpecName "kube-api-access-lzw8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:35:50.725014 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.724989 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec262afd-0465-420e-a94d-d768905bf5e3-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ec262afd-0465-420e-a94d-d768905bf5e3" (UID: "ec262afd-0465-420e-a94d-d768905bf5e3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:35:50.814718 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.814640 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec262afd-0465-420e-a94d-d768905bf5e3-tls-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:35:50.814718 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:50.814672 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzw8m\" (UniqueName: \"kubernetes.io/projected/ec262afd-0465-420e-a94d-d768905bf5e3-kube-api-access-lzw8m\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:35:51.224788 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.224742 2559 generic.go:356] "Generic (PLEG): container finished" podID="ec262afd-0465-420e-a94d-d768905bf5e3" containerID="13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f" exitCode=0 Jul 15 12:35:51.224788 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.224776 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cc9685df-pnv8f" Jul 15 12:35:51.225371 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.224825 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cc9685df-pnv8f" event={"ID":"ec262afd-0465-420e-a94d-d768905bf5e3","Type":"ContainerDied","Data":"13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f"} Jul 15 12:35:51.225371 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.224866 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cc9685df-pnv8f" event={"ID":"ec262afd-0465-420e-a94d-d768905bf5e3","Type":"ContainerDied","Data":"48759c0178a1a80b228a4800cb5f38ccf3c228d68855716465c7cc503a517af9"} Jul 15 12:35:51.225371 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.224888 2559 scope.go:117] "RemoveContainer" containerID="13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f" Jul 15 12:35:51.235172 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.235153 2559 scope.go:117] "RemoveContainer" containerID="13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f" Jul 15 12:35:51.235448 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:35:51.235430 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f\": container with ID starting with 13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f not found: ID does not exist" containerID="13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f" Jul 15 12:35:51.235523 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.235461 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f"} err="failed to get container status \"13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f\": rpc error: code = NotFound desc = could not find container \"13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f\": container with ID starting with 13f970a646a716d5607c9adef500a7f043873efc7bafdbd2ac72805c6971799f not found: ID does not exist" Jul 15 12:35:51.251637 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.251615 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54cc9685df-pnv8f"] Jul 15 12:35:51.258895 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:51.258869 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54cc9685df-pnv8f"] Jul 15 12:35:52.111451 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:35:52.111419 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec262afd-0465-420e-a94d-d768905bf5e3" path="/var/lib/kubelet/pods/ec262afd-0465-420e-a94d-d768905bf5e3/volumes" Jul 15 12:37:01.618495 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.618403 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-28x87"] Jul 15 12:37:01.619117 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.618620 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fc57dd599-28x87" podUID="0af794a5-6517-434e-b4d2-b9b51e0861b5" containerName="manager" containerID="cri-o://16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50" gracePeriod=30 Jul 15 12:37:01.882689 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.882630 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:37:01.958037 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.958010 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0af794a5-6517-434e-b4d2-b9b51e0861b5-webhook-cert\") pod \"0af794a5-6517-434e-b4d2-b9b51e0861b5\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " Jul 15 12:37:01.958200 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.958143 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gj6z\" (UniqueName: \"kubernetes.io/projected/0af794a5-6517-434e-b4d2-b9b51e0861b5-kube-api-access-5gj6z\") pod \"0af794a5-6517-434e-b4d2-b9b51e0861b5\" (UID: \"0af794a5-6517-434e-b4d2-b9b51e0861b5\") " Jul 15 12:37:01.960070 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.960041 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af794a5-6517-434e-b4d2-b9b51e0861b5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0af794a5-6517-434e-b4d2-b9b51e0861b5" (UID: "0af794a5-6517-434e-b4d2-b9b51e0861b5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:37:01.960176 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:01.960061 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af794a5-6517-434e-b4d2-b9b51e0861b5-kube-api-access-5gj6z" (OuterVolumeSpecName: "kube-api-access-5gj6z") pod "0af794a5-6517-434e-b4d2-b9b51e0861b5" (UID: "0af794a5-6517-434e-b4d2-b9b51e0861b5"). InnerVolumeSpecName "kube-api-access-5gj6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:37:02.059657 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.059625 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gj6z\" (UniqueName: \"kubernetes.io/projected/0af794a5-6517-434e-b4d2-b9b51e0861b5-kube-api-access-5gj6z\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:37:02.059657 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.059659 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0af794a5-6517-434e-b4d2-b9b51e0861b5-webhook-cert\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:37:02.656104 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.656073 2559 generic.go:356] "Generic (PLEG): container finished" podID="0af794a5-6517-434e-b4d2-b9b51e0861b5" containerID="16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50" exitCode=0 Jul 15 12:37:02.656510 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.656127 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc57dd599-28x87" Jul 15 12:37:02.656510 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.656162 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc57dd599-28x87" event={"ID":"0af794a5-6517-434e-b4d2-b9b51e0861b5","Type":"ContainerDied","Data":"16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50"} Jul 15 12:37:02.656510 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.656215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc57dd599-28x87" event={"ID":"0af794a5-6517-434e-b4d2-b9b51e0861b5","Type":"ContainerDied","Data":"6e81ea4bd7be3a90010e263f5c6ecdef1a3f9982d49be5ed474ed90c80d4fbdd"} Jul 15 12:37:02.656510 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.656237 2559 scope.go:117] "RemoveContainer" containerID="16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50" Jul 15 12:37:02.666351 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.666330 2559 scope.go:117] "RemoveContainer" containerID="16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50" Jul 15 12:37:02.666638 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:37:02.666616 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50\": container with ID starting with 16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50 not found: ID does not exist" containerID="16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50" Jul 15 12:37:02.666721 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.666649 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50"} err="failed to get container status \"16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50\": rpc error: code = NotFound desc = could not find container \"16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50\": container with ID starting with 16306bcbd67c2c39fb621a3a7b9b4798c0345026a30938d233f9b15f2f609c50 not found: ID does not exist" Jul 15 12:37:02.674672 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.674649 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-28x87"] Jul 15 12:37:02.682617 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:02.682595 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-28x87"] Jul 15 12:37:03.544335 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.544297 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fc57dd599-d9rk9"] Jul 15 12:37:03.544875 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.544856 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec262afd-0465-420e-a94d-d768905bf5e3" containerName="authorino" Jul 15 12:37:03.544875 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.544876 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec262afd-0465-420e-a94d-d768905bf5e3" containerName="authorino" Jul 15 12:37:03.545051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.544902 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af794a5-6517-434e-b4d2-b9b51e0861b5" containerName="manager" Jul 15 12:37:03.545051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.544910 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af794a5-6517-434e-b4d2-b9b51e0861b5" containerName="manager" Jul 15 12:37:03.545051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.545033 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec262afd-0465-420e-a94d-d768905bf5e3" containerName="authorino" Jul 15 12:37:03.545051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.545050 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af794a5-6517-434e-b4d2-b9b51e0861b5" containerName="manager" Jul 15 12:37:03.549563 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.549545 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.553509 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.553491 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 12:37:03.553590 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.553495 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6rwqv\"" Jul 15 12:37:03.553659 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.553498 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 12:37:03.558699 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.558677 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-d9rk9"] Jul 15 12:37:03.674721 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.674687 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s75d\" (UniqueName: \"kubernetes.io/projected/87e69da8-f9ac-45fd-befa-4a707a06ff6b-kube-api-access-7s75d\") pod \"maas-controller-7fc57dd599-d9rk9\" (UID: \"87e69da8-f9ac-45fd-befa-4a707a06ff6b\") " pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.675125 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.674747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87e69da8-f9ac-45fd-befa-4a707a06ff6b-webhook-cert\") pod \"maas-controller-7fc57dd599-d9rk9\" (UID: \"87e69da8-f9ac-45fd-befa-4a707a06ff6b\") " pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.775509 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.775474 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s75d\" (UniqueName: \"kubernetes.io/projected/87e69da8-f9ac-45fd-befa-4a707a06ff6b-kube-api-access-7s75d\") pod \"maas-controller-7fc57dd599-d9rk9\" (UID: \"87e69da8-f9ac-45fd-befa-4a707a06ff6b\") " pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.775654 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.775517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87e69da8-f9ac-45fd-befa-4a707a06ff6b-webhook-cert\") pod \"maas-controller-7fc57dd599-d9rk9\" (UID: \"87e69da8-f9ac-45fd-befa-4a707a06ff6b\") " pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.777987 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.777970 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87e69da8-f9ac-45fd-befa-4a707a06ff6b-webhook-cert\") pod \"maas-controller-7fc57dd599-d9rk9\" (UID: \"87e69da8-f9ac-45fd-befa-4a707a06ff6b\") " pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.784198 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.784181 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s75d\" (UniqueName: \"kubernetes.io/projected/87e69da8-f9ac-45fd-befa-4a707a06ff6b-kube-api-access-7s75d\") pod \"maas-controller-7fc57dd599-d9rk9\" (UID: \"87e69da8-f9ac-45fd-befa-4a707a06ff6b\") " pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:03.860981 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:03.860920 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:04.004598 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:04.004573 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc57dd599-d9rk9"] Jul 15 12:37:04.006157 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:37:04.006129 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e69da8_f9ac_45fd_befa_4a707a06ff6b.slice/crio-100df8f13955a1d5657bf1c1508f14f03dbe7c6d4bad86a45a73fdc732767d46 WatchSource:0}: Error finding container 100df8f13955a1d5657bf1c1508f14f03dbe7c6d4bad86a45a73fdc732767d46: Status 404 returned error can't find the container with id 100df8f13955a1d5657bf1c1508f14f03dbe7c6d4bad86a45a73fdc732767d46 Jul 15 12:37:04.110421 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:04.110388 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af794a5-6517-434e-b4d2-b9b51e0861b5" path="/var/lib/kubelet/pods/0af794a5-6517-434e-b4d2-b9b51e0861b5/volumes" Jul 15 12:37:04.670915 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:04.670852 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" event={"ID":"87e69da8-f9ac-45fd-befa-4a707a06ff6b","Type":"ContainerStarted","Data":"51dc3d5e3bbb98729bbb1fa302c4aaf7d87606b59185754bd45aad9754138fa1"} Jul 15 12:37:04.670915 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:04.670918 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:37:04.671188 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:04.670941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" event={"ID":"87e69da8-f9ac-45fd-befa-4a707a06ff6b","Type":"ContainerStarted","Data":"100df8f13955a1d5657bf1c1508f14f03dbe7c6d4bad86a45a73fdc732767d46"} Jul 15 12:37:04.691197 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:04.691151 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" podStartSLOduration=1.108638403 podStartE2EDuration="1.691136119s" totalImagesPullingTime="582.497716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:37:04.688325888 +0000 UTC m=+825.239085440" watchObservedRunningTime="2026-07-15 12:37:04.691136119 +0000 UTC m=+825.241895651" Jul 15 12:37:15.680148 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:37:15.680114 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fc57dd599-d9rk9" Jul 15 12:38:19.982896 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:38:19.982823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:38:19.983945 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:38:19.983924 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:42:04.054220 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:04.054183 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h"] Jul 15 12:42:04.054943 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:04.054390 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" podUID="6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" containerName="manager" containerID="cri-o://697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee" gracePeriod=10 Jul 15 12:42:07.608556 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.608534 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:42:07.635934 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.635881 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-extensions-socket-volume\") pod \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " Jul 15 12:42:07.635934 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.635923 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-849cg\" (UniqueName: \"kubernetes.io/projected/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-kube-api-access-849cg\") pod \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\" (UID: \"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5\") " Jul 15 12:42:07.636273 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.636249 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" (UID: "6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:42:07.638145 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.638123 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-kube-api-access-849cg" (OuterVolumeSpecName: "kube-api-access-849cg") pod "6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" (UID: "6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5"). InnerVolumeSpecName "kube-api-access-849cg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:42:07.736924 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.736898 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-extensions-socket-volume\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:42:07.736924 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:07.736922 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-849cg\" (UniqueName: \"kubernetes.io/projected/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5-kube-api-access-849cg\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:42:08.462281 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.462250 2559 generic.go:356] "Generic (PLEG): container finished" podID="6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" containerID="697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee" exitCode=0 Jul 15 12:42:08.462459 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.462300 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" Jul 15 12:42:08.462459 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.462335 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" event={"ID":"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5","Type":"ContainerDied","Data":"697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee"} Jul 15 12:42:08.462459 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.462391 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h" event={"ID":"6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5","Type":"ContainerDied","Data":"1ec459b90ed7792fc849208f7b59c42ef1fc207f9903684dab658396b102d82a"} Jul 15 12:42:08.462459 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.462415 2559 scope.go:117] "RemoveContainer" containerID="697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee" Jul 15 12:42:08.473484 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.473281 2559 scope.go:117] "RemoveContainer" containerID="697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee" Jul 15 12:42:08.473561 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:42:08.473545 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee\": container with ID starting with 697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee not found: ID does not exist" containerID="697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee" Jul 15 12:42:08.473612 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.473568 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee"} err="failed to get container status \"697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee\": rpc error: code = NotFound desc = could not find container \"697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee\": container with ID starting with 697e82c53e8145e2bc77ea908dd9b64527aa2a50d13dd34cbe9409d780f984ee not found: ID does not exist" Jul 15 12:42:08.494817 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.494766 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h"] Jul 15 12:42:08.505308 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:08.505289 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kfq5h"] Jul 15 12:42:10.119840 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:42:10.119784 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" path="/var/lib/kubelet/pods/6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5/volumes" Jul 15 12:43:05.374024 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.373952 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5"] Jul 15 12:43:05.374462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.374368 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" containerName="manager" Jul 15 12:43:05.374462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.374378 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" containerName="manager" Jul 15 12:43:05.374462 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.374462 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee6d13d-3ddd-4cd0-bb56-5db5250a8ae5" containerName="manager" Jul 15 12:43:05.377555 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.377540 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.381067 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.381040 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mkb9k\"" Jul 15 12:43:05.389129 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.389109 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cc9cc6f-001e-4852-8c91-233785c0d043-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dpgl5\" (UID: \"0cc9cc6f-001e-4852-8c91-233785c0d043\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.389240 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.389151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjbxr\" (UniqueName: \"kubernetes.io/projected/0cc9cc6f-001e-4852-8c91-233785c0d043-kube-api-access-vjbxr\") pod \"kuadrant-operator-controller-manager-86878c769f-dpgl5\" (UID: \"0cc9cc6f-001e-4852-8c91-233785c0d043\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.410157 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.410135 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5"] Jul 15 12:43:05.489998 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.489974 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cc9cc6f-001e-4852-8c91-233785c0d043-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dpgl5\" (UID: \"0cc9cc6f-001e-4852-8c91-233785c0d043\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.490095 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.490017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjbxr\" (UniqueName: \"kubernetes.io/projected/0cc9cc6f-001e-4852-8c91-233785c0d043-kube-api-access-vjbxr\") pod \"kuadrant-operator-controller-manager-86878c769f-dpgl5\" (UID: \"0cc9cc6f-001e-4852-8c91-233785c0d043\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.490314 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.490295 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cc9cc6f-001e-4852-8c91-233785c0d043-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dpgl5\" (UID: \"0cc9cc6f-001e-4852-8c91-233785c0d043\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.502529 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.502505 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjbxr\" (UniqueName: \"kubernetes.io/projected/0cc9cc6f-001e-4852-8c91-233785c0d043-kube-api-access-vjbxr\") pod \"kuadrant-operator-controller-manager-86878c769f-dpgl5\" (UID: \"0cc9cc6f-001e-4852-8c91-233785c0d043\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.688198 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.688152 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:05.845839 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:05.845786 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5"] Jul 15 12:43:05.846203 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:43:05.846182 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc9cc6f_001e_4852_8c91_233785c0d043.slice/crio-4d74c81e2c206cd5b7818c2d126344f3f67871e76a6d390193de237dc2359944 WatchSource:0}: Error finding container 4d74c81e2c206cd5b7818c2d126344f3f67871e76a6d390193de237dc2359944: Status 404 returned error can't find the container with id 4d74c81e2c206cd5b7818c2d126344f3f67871e76a6d390193de237dc2359944 Jul 15 12:43:06.801051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:06.800948 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" event={"ID":"0cc9cc6f-001e-4852-8c91-233785c0d043","Type":"ContainerStarted","Data":"f05ded4e33f30dd669ef81ceea45e8cd58eb66abd8623a8cf178bf4c84fbf978"} Jul 15 12:43:06.801051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:06.801004 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" event={"ID":"0cc9cc6f-001e-4852-8c91-233785c0d043","Type":"ContainerStarted","Data":"4d74c81e2c206cd5b7818c2d126344f3f67871e76a6d390193de237dc2359944"} Jul 15 12:43:06.801051 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:06.801053 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:06.831104 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:06.831056 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" podStartSLOduration=1.83103774 podStartE2EDuration="1.83103774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:43:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:43:06.828919998 +0000 UTC m=+1187.379679547" watchObservedRunningTime="2026-07-15 12:43:06.83103774 +0000 UTC m=+1187.381797272" Jul 15 12:43:17.809772 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:17.809695 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dpgl5" Jul 15 12:43:20.027646 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:20.027623 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:43:20.030597 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:43:20.030565 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:48:20.067019 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:48:20.066989 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:48:20.072650 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:48:20.072628 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:50:52.846746 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.846718 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m"] Jul 15 12:50:52.850354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.850335 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:52.853470 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.853454 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 12:50:52.860616 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.860590 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m"] Jul 15 12:50:52.872602 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.872581 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c568951-1ab3-41e3-8bc1-50ba509f67b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m\" (UID: \"0c568951-1ab3-41e3-8bc1-50ba509f67b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:52.973343 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.973312 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c568951-1ab3-41e3-8bc1-50ba509f67b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m\" (UID: \"0c568951-1ab3-41e3-8bc1-50ba509f67b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:52.973937 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:52.973918 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c568951-1ab3-41e3-8bc1-50ba509f67b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m\" (UID: \"0c568951-1ab3-41e3-8bc1-50ba509f67b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:53.161594 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:53.161541 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:53.506991 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:53.506968 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m"] Jul 15 12:50:53.508685 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:50:53.508658 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c568951_1ab3_41e3_8bc1_50ba509f67b6.slice/crio-839b04e59c5d05b13de5e440c7d505a122bca907a3b669e8c9c37e891d60fc83 WatchSource:0}: Error finding container 839b04e59c5d05b13de5e440c7d505a122bca907a3b669e8c9c37e891d60fc83: Status 404 returned error can't find the container with id 839b04e59c5d05b13de5e440c7d505a122bca907a3b669e8c9c37e891d60fc83 Jul 15 12:50:53.510530 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:53.510512 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:50:53.593658 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:53.593630 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" event={"ID":"0c568951-1ab3-41e3-8bc1-50ba509f67b6","Type":"ContainerStarted","Data":"839b04e59c5d05b13de5e440c7d505a122bca907a3b669e8c9c37e891d60fc83"} Jul 15 12:50:57.618724 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:57.618690 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c568951-1ab3-41e3-8bc1-50ba509f67b6" containerID="f98485705972728f9cba248becb83c5cc632984d95b46769dd5d38eecd305de7" exitCode=0 Jul 15 12:50:57.619132 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:57.618742 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" event={"ID":"0c568951-1ab3-41e3-8bc1-50ba509f67b6","Type":"ContainerDied","Data":"f98485705972728f9cba248becb83c5cc632984d95b46769dd5d38eecd305de7"} Jul 15 12:50:58.770426 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:58.770401 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:58.823013 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:58.822990 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c568951-1ab3-41e3-8bc1-50ba509f67b6-maas-api-service-ca\") pod \"0c568951-1ab3-41e3-8bc1-50ba509f67b6\" (UID: \"0c568951-1ab3-41e3-8bc1-50ba509f67b6\") " Jul 15 12:50:58.823299 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:58.823280 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c568951-1ab3-41e3-8bc1-50ba509f67b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c568951-1ab3-41e3-8bc1-50ba509f67b6" (UID: "0c568951-1ab3-41e3-8bc1-50ba509f67b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:50:58.924381 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:58.924318 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c568951-1ab3-41e3-8bc1-50ba509f67b6-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:50:59.633406 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:59.633381 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" Jul 15 12:50:59.633587 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:59.633412 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m" event={"ID":"0c568951-1ab3-41e3-8bc1-50ba509f67b6","Type":"ContainerDied","Data":"839b04e59c5d05b13de5e440c7d505a122bca907a3b669e8c9c37e891d60fc83"} Jul 15 12:50:59.633587 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:50:59.633458 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="839b04e59c5d05b13de5e440c7d505a122bca907a3b669e8c9c37e891d60fc83" Jul 15 12:52:56.981594 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.981557 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh"] Jul 15 12:52:56.982255 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.982235 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c568951-1ab3-41e3-8bc1-50ba509f67b6" containerName="revoke-keys" Jul 15 12:52:56.982333 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.982258 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c568951-1ab3-41e3-8bc1-50ba509f67b6" containerName="revoke-keys" Jul 15 12:52:56.982398 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.982359 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c568951-1ab3-41e3-8bc1-50ba509f67b6" containerName="revoke-keys" Jul 15 12:52:56.986002 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.985979 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:52:56.989149 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.989129 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 12:52:56.992951 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:56.992929 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh"] Jul 15 12:52:57.090104 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:57.090080 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5479d443-4b5f-4a89-b949-eddbba833e16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh\" (UID: \"5479d443-4b5f-4a89-b949-eddbba833e16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:52:57.191421 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:57.191390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5479d443-4b5f-4a89-b949-eddbba833e16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh\" (UID: \"5479d443-4b5f-4a89-b949-eddbba833e16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:52:57.191924 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:57.191907 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5479d443-4b5f-4a89-b949-eddbba833e16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh\" (UID: \"5479d443-4b5f-4a89-b949-eddbba833e16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:52:57.299304 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:57.299229 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:52:57.652283 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:57.652257 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh"] Jul 15 12:52:57.652735 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:52:57.652710 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5479d443_4b5f_4a89_b949_eddbba833e16.slice/crio-1e32114a7674ca595a3bbd1e14a27877493b8e46bfd4ae0a8864a42dc4a1b1d1 WatchSource:0}: Error finding container 1e32114a7674ca595a3bbd1e14a27877493b8e46bfd4ae0a8864a42dc4a1b1d1: Status 404 returned error can't find the container with id 1e32114a7674ca595a3bbd1e14a27877493b8e46bfd4ae0a8864a42dc4a1b1d1 Jul 15 12:52:58.326753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:58.326671 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerStarted","Data":"e8b961410fa10a52da238e3b61531d233b2a9a90725e0dd1a4d908be228f9d80"} Jul 15 12:52:58.326753 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:58.326719 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerStarted","Data":"1e32114a7674ca595a3bbd1e14a27877493b8e46bfd4ae0a8864a42dc4a1b1d1"} Jul 15 12:52:58.348230 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:58.348189 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" podStartSLOduration=2.348174634 podStartE2EDuration="2.348174634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:52:58.345221563 +0000 UTC m=+1778.895981094" watchObservedRunningTime="2026-07-15 12:52:58.348174634 +0000 UTC m=+1778.898934166" Jul 15 12:52:59.334100 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:59.334064 2559 generic.go:356] "Generic (PLEG): container finished" podID="5479d443-4b5f-4a89-b949-eddbba833e16" containerID="e8b961410fa10a52da238e3b61531d233b2a9a90725e0dd1a4d908be228f9d80" exitCode=7 Jul 15 12:52:59.334458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:59.334138 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerDied","Data":"e8b961410fa10a52da238e3b61531d233b2a9a90725e0dd1a4d908be228f9d80"} Jul 15 12:52:59.334458 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:52:59.334325 2559 scope.go:117] "RemoveContainer" containerID="e8b961410fa10a52da238e3b61531d233b2a9a90725e0dd1a4d908be228f9d80" Jul 15 12:53:00.342679 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:00.342643 2559 generic.go:356] "Generic (PLEG): container finished" podID="5479d443-4b5f-4a89-b949-eddbba833e16" containerID="bc5b6c9d7311e66fd5577bce769ef1ea78ed608da21da6aee744a697b1ad0308" exitCode=7 Jul 15 12:53:00.343146 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:00.342694 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerDied","Data":"bc5b6c9d7311e66fd5577bce769ef1ea78ed608da21da6aee744a697b1ad0308"} Jul 15 12:53:00.343146 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:00.342753 2559 scope.go:117] "RemoveContainer" containerID="e8b961410fa10a52da238e3b61531d233b2a9a90725e0dd1a4d908be228f9d80" Jul 15 12:53:00.343146 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:00.342905 2559 scope.go:117] "RemoveContainer" containerID="bc5b6c9d7311e66fd5577bce769ef1ea78ed608da21da6aee744a697b1ad0308" Jul 15 12:53:00.343146 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:53:00.343140 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh_odh-ai-gateway-infra(5479d443-4b5f-4a89-b949-eddbba833e16)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" Jul 15 12:53:01.350354 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:01.350326 2559 scope.go:117] "RemoveContainer" containerID="bc5b6c9d7311e66fd5577bce769ef1ea78ed608da21da6aee744a697b1ad0308" Jul 15 12:53:01.350727 ip-10-0-142-131 kubenswrapper[2559]: E0715 12:53:01.350511 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh_odh-ai-gateway-infra(5479d443-4b5f-4a89-b949-eddbba833e16)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" Jul 15 12:53:13.104071 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:13.103995 2559 scope.go:117] "RemoveContainer" containerID="bc5b6c9d7311e66fd5577bce769ef1ea78ed608da21da6aee744a697b1ad0308" Jul 15 12:53:13.428270 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:13.428247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerStarted","Data":"ca8865bb75323a784c99cf75362591e0552a4618cfffd7e5a40b77411e7e082f"} Jul 15 12:53:14.443791 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:14.443758 2559 generic.go:356] "Generic (PLEG): container finished" podID="5479d443-4b5f-4a89-b949-eddbba833e16" containerID="ca8865bb75323a784c99cf75362591e0552a4618cfffd7e5a40b77411e7e082f" exitCode=0 Jul 15 12:53:14.444217 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:14.443831 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerDied","Data":"ca8865bb75323a784c99cf75362591e0552a4618cfffd7e5a40b77411e7e082f"} Jul 15 12:53:14.444217 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:14.443915 2559 scope.go:117] "RemoveContainer" containerID="bc5b6c9d7311e66fd5577bce769ef1ea78ed608da21da6aee744a697b1ad0308" Jul 15 12:53:14.458373 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:14.458350 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh"] Jul 15 12:53:15.594870 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:15.594845 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:53:15.768026 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:15.767999 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5479d443-4b5f-4a89-b949-eddbba833e16-maas-api-service-ca\") pod \"5479d443-4b5f-4a89-b949-eddbba833e16\" (UID: \"5479d443-4b5f-4a89-b949-eddbba833e16\") " Jul 15 12:53:15.768326 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:15.768306 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5479d443-4b5f-4a89-b949-eddbba833e16-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5479d443-4b5f-4a89-b949-eddbba833e16" (UID: "5479d443-4b5f-4a89-b949-eddbba833e16"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:53:15.868914 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:15.868886 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5479d443-4b5f-4a89-b949-eddbba833e16-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:53:16.458888 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:16.458870 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" Jul 15 12:53:16.459028 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:16.458862 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh" event={"ID":"5479d443-4b5f-4a89-b949-eddbba833e16","Type":"ContainerDied","Data":"1e32114a7674ca595a3bbd1e14a27877493b8e46bfd4ae0a8864a42dc4a1b1d1"} Jul 15 12:53:16.459028 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:16.458997 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e32114a7674ca595a3bbd1e14a27877493b8e46bfd4ae0a8864a42dc4a1b1d1" Jul 15 12:53:16.498079 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:16.498051 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh"] Jul 15 12:53:16.504862 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:16.504833 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e27a08d-lm6gh"] Jul 15 12:53:18.111529 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:18.111501 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" path="/var/lib/kubelet/pods/5479d443-4b5f-4a89-b949-eddbba833e16/volumes" Jul 15 12:53:20.109910 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:20.109769 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:53:20.132328 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:53:20.116474 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:56:08.487956 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.487918 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z"] Jul 15 12:56:08.488470 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.488433 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:56:08.488470 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.488447 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:56:08.488470 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.488463 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:56:08.488575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.488473 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:56:08.488575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.488559 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:56:08.488575 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.488569 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:56:08.491496 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.491477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:08.495208 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.495187 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 12:56:08.503730 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.503711 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z"] Jul 15 12:56:08.634432 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.634406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c938f35-4879-417b-9e5a-9f7da205e91c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8635587b-pmw7z\" (UID: \"5c938f35-4879-417b-9e5a-9f7da205e91c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:08.735185 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.735164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c938f35-4879-417b-9e5a-9f7da205e91c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8635587b-pmw7z\" (UID: \"5c938f35-4879-417b-9e5a-9f7da205e91c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:08.735701 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.735684 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c938f35-4879-417b-9e5a-9f7da205e91c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8635587b-pmw7z\" (UID: \"5c938f35-4879-417b-9e5a-9f7da205e91c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:08.802252 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.802200 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:08.959277 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:08.959249 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z"] Jul 15 12:56:08.960348 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:56:08.960325 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c938f35_4879_417b_9e5a_9f7da205e91c.slice/crio-cfa480daf6e4249cb3ef58810cbbbc67f9c2a510f1e782b527600d93a55a4725 WatchSource:0}: Error finding container cfa480daf6e4249cb3ef58810cbbbc67f9c2a510f1e782b527600d93a55a4725: Status 404 returned error can't find the container with id cfa480daf6e4249cb3ef58810cbbbc67f9c2a510f1e782b527600d93a55a4725 Jul 15 12:56:09.500538 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:09.500503 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" event={"ID":"5c938f35-4879-417b-9e5a-9f7da205e91c","Type":"ContainerStarted","Data":"7efc6e0053b2c07b8bf25cac01e39f852c967d789cf0975d12ac8c5444caa520"} Jul 15 12:56:09.501029 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:09.500565 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" event={"ID":"5c938f35-4879-417b-9e5a-9f7da205e91c","Type":"ContainerStarted","Data":"cfa480daf6e4249cb3ef58810cbbbc67f9c2a510f1e782b527600d93a55a4725"} Jul 15 12:56:09.528588 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:09.528545 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" podStartSLOduration=1.528530784 podStartE2EDuration="1.528530784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:56:09.527759268 +0000 UTC m=+1970.078518824" watchObservedRunningTime="2026-07-15 12:56:09.528530784 +0000 UTC m=+1970.079290370" Jul 15 12:56:10.509160 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:10.509126 2559 generic.go:356] "Generic (PLEG): container finished" podID="5c938f35-4879-417b-9e5a-9f7da205e91c" containerID="7efc6e0053b2c07b8bf25cac01e39f852c967d789cf0975d12ac8c5444caa520" exitCode=0 Jul 15 12:56:10.509582 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:10.509219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" event={"ID":"5c938f35-4879-417b-9e5a-9f7da205e91c","Type":"ContainerDied","Data":"7efc6e0053b2c07b8bf25cac01e39f852c967d789cf0975d12ac8c5444caa520"} Jul 15 12:56:11.662694 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:11.662670 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:11.762869 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:11.762848 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c938f35-4879-417b-9e5a-9f7da205e91c-maas-api-service-ca\") pod \"5c938f35-4879-417b-9e5a-9f7da205e91c\" (UID: \"5c938f35-4879-417b-9e5a-9f7da205e91c\") " Jul 15 12:56:11.763211 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:11.763192 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c938f35-4879-417b-9e5a-9f7da205e91c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c938f35-4879-417b-9e5a-9f7da205e91c" (UID: "5c938f35-4879-417b-9e5a-9f7da205e91c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:56:11.864074 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:11.864018 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c938f35-4879-417b-9e5a-9f7da205e91c-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:56:12.524848 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:12.524817 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" Jul 15 12:56:12.524848 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:12.524836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8635587b-pmw7z" event={"ID":"5c938f35-4879-417b-9e5a-9f7da205e91c","Type":"ContainerDied","Data":"cfa480daf6e4249cb3ef58810cbbbc67f9c2a510f1e782b527600d93a55a4725"} Jul 15 12:56:12.525049 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:56:12.524886 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfa480daf6e4249cb3ef58810cbbbc67f9c2a510f1e782b527600d93a55a4725" Jul 15 12:57:30.756625 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.756596 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt"] Jul 15 12:57:30.757099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.757014 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c938f35-4879-417b-9e5a-9f7da205e91c" containerName="revoke-keys" Jul 15 12:57:30.757099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.757024 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c938f35-4879-417b-9e5a-9f7da205e91c" containerName="revoke-keys" Jul 15 12:57:30.757099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.757058 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:57:30.757099 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.757064 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:57:30.757271 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.757136 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5479d443-4b5f-4a89-b949-eddbba833e16" containerName="revoke-keys" Jul 15 12:57:30.757271 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.757148 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c938f35-4879-417b-9e5a-9f7da205e91c" containerName="revoke-keys" Jul 15 12:57:30.760250 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.760232 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:30.766026 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.766007 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 12:57:30.782036 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.782011 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt"] Jul 15 12:57:30.834147 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.834119 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20c08b1-4490-4daf-9833-4dbcf2b3780c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt\" (UID: \"e20c08b1-4490-4daf-9833-4dbcf2b3780c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:30.935502 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.935470 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20c08b1-4490-4daf-9833-4dbcf2b3780c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt\" (UID: \"e20c08b1-4490-4daf-9833-4dbcf2b3780c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:30.936192 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:30.936172 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20c08b1-4490-4daf-9833-4dbcf2b3780c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt\" (UID: \"e20c08b1-4490-4daf-9833-4dbcf2b3780c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:31.071847 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:31.071751 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:31.217194 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:31.217163 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt"] Jul 15 12:57:31.217443 ip-10-0-142-131 kubenswrapper[2559]: W0715 12:57:31.217419 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20c08b1_4490_4daf_9833_4dbcf2b3780c.slice/crio-67d2a7bfb05273dcc1e12f9f07c3b9024a60b569f6795c054e927c653b8f010e WatchSource:0}: Error finding container 67d2a7bfb05273dcc1e12f9f07c3b9024a60b569f6795c054e927c653b8f010e: Status 404 returned error can't find the container with id 67d2a7bfb05273dcc1e12f9f07c3b9024a60b569f6795c054e927c653b8f010e Jul 15 12:57:32.017719 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:32.017688 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" event={"ID":"e20c08b1-4490-4daf-9833-4dbcf2b3780c","Type":"ContainerStarted","Data":"84d35f3dfd4c34bb69e95e37b0e77f748996a39b0d28b016d4d30840237c5ea6"} Jul 15 12:57:32.018126 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:32.017741 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" event={"ID":"e20c08b1-4490-4daf-9833-4dbcf2b3780c","Type":"ContainerStarted","Data":"67d2a7bfb05273dcc1e12f9f07c3b9024a60b569f6795c054e927c653b8f010e"} Jul 15 12:57:32.053604 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:32.053554 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" podStartSLOduration=2.053535561 podStartE2EDuration="2.053535561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:57:32.051179698 +0000 UTC m=+2052.601939243" watchObservedRunningTime="2026-07-15 12:57:32.053535561 +0000 UTC m=+2052.604295095" Jul 15 12:57:33.024838 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:33.024784 2559 generic.go:356] "Generic (PLEG): container finished" podID="e20c08b1-4490-4daf-9833-4dbcf2b3780c" containerID="84d35f3dfd4c34bb69e95e37b0e77f748996a39b0d28b016d4d30840237c5ea6" exitCode=0 Jul 15 12:57:33.025210 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:33.024872 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" event={"ID":"e20c08b1-4490-4daf-9833-4dbcf2b3780c","Type":"ContainerDied","Data":"84d35f3dfd4c34bb69e95e37b0e77f748996a39b0d28b016d4d30840237c5ea6"} Jul 15 12:57:34.175576 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:34.175553 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:34.265219 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:34.265194 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20c08b1-4490-4daf-9833-4dbcf2b3780c-maas-api-service-ca\") pod \"e20c08b1-4490-4daf-9833-4dbcf2b3780c\" (UID: \"e20c08b1-4490-4daf-9833-4dbcf2b3780c\") " Jul 15 12:57:34.265894 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:34.265553 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20c08b1-4490-4daf-9833-4dbcf2b3780c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e20c08b1-4490-4daf-9833-4dbcf2b3780c" (UID: "e20c08b1-4490-4daf-9833-4dbcf2b3780c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:57:34.367078 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:34.367012 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20c08b1-4490-4daf-9833-4dbcf2b3780c-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 12:57:35.037936 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:35.037910 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" Jul 15 12:57:35.037936 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:35.037914 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt" event={"ID":"e20c08b1-4490-4daf-9833-4dbcf2b3780c","Type":"ContainerDied","Data":"67d2a7bfb05273dcc1e12f9f07c3b9024a60b569f6795c054e927c653b8f010e"} Jul 15 12:57:35.038145 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:57:35.037957 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67d2a7bfb05273dcc1e12f9f07c3b9024a60b569f6795c054e927c653b8f010e" Jul 15 12:58:20.150132 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:58:20.150037 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:58:20.162906 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:58:20.162889 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 12:59:20.238344 ip-10-0-142-131 kubenswrapper[2559]: I0715 12:59:20.238277 2559 scope.go:117] "RemoveContainer" containerID="ca8865bb75323a784c99cf75362591e0552a4618cfffd7e5a40b77411e7e082f" Jul 15 13:00:00.145848 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.145812 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn"] Jul 15 13:00:00.146326 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.146266 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20c08b1-4490-4daf-9833-4dbcf2b3780c" containerName="revoke-keys" Jul 15 13:00:00.146326 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.146278 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20c08b1-4490-4daf-9833-4dbcf2b3780c" containerName="revoke-keys" Jul 15 13:00:00.146400 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.146365 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20c08b1-4490-4daf-9833-4dbcf2b3780c" containerName="revoke-keys" Jul 15 13:00:00.149673 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.149653 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" Jul 15 13:00:00.151738 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.151717 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd"] Jul 15 13:00:00.152907 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.152881 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 13:00:00.155139 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.155118 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" Jul 15 13:00:00.160786 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.160768 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" Jul 15 13:00:00.167909 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.167887 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" Jul 15 13:00:00.170196 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.170172 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn"] Jul 15 13:00:00.177290 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.177259 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd"] Jul 15 13:00:00.533475 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.533446 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd"] Jul 15 13:00:00.538409 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:00:00.538381 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b1d27a_f3fd_4adc_bdb9_d0c493008350.slice/crio-5e8605d81772ca4f526b1b68d93784430ab4f9af8b3e31127f2b1f75db0968a7 WatchSource:0}: Error finding container 5e8605d81772ca4f526b1b68d93784430ab4f9af8b3e31127f2b1f75db0968a7: Status 404 returned error can't find the container with id 5e8605d81772ca4f526b1b68d93784430ab4f9af8b3e31127f2b1f75db0968a7 Jul 15 13:00:00.563797 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:00:00.563767 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af0c3e4_6f23_4c81_97eb_7b0847ef285c.slice/crio-361cdbb08693d5f75b725627974c202af84918f6108d6515a23bdf88db3a544f WatchSource:0}: Error finding container 361cdbb08693d5f75b725627974c202af84918f6108d6515a23bdf88db3a544f: Status 404 returned error can't find the container with id 361cdbb08693d5f75b725627974c202af84918f6108d6515a23bdf88db3a544f Jul 15 13:00:00.564133 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.564100 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn"] Jul 15 13:00:00.923169 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.923141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerStarted","Data":"1327b73e6bf93d156e322b917e5eb4f9979fa36a2874d0c338fd27db7e14fc42"} Jul 15 13:00:00.923295 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.923196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerStarted","Data":"5e8605d81772ca4f526b1b68d93784430ab4f9af8b3e31127f2b1f75db0968a7"} Jul 15 13:00:00.924414 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.924389 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" event={"ID":"0af0c3e4-6f23-4c81-97eb-7b0847ef285c","Type":"ContainerStarted","Data":"361cdbb08693d5f75b725627974c202af84918f6108d6515a23bdf88db3a544f"} Jul 15 13:00:00.949888 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:00.949847 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" podStartSLOduration=0.949831341 podStartE2EDuration="949.831341ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:00:00.946987906 +0000 UTC m=+2201.497747439" watchObservedRunningTime="2026-07-15 13:00:00.949831341 +0000 UTC m=+2201.500590872" Jul 15 13:00:01.932914 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:01.932878 2559 generic.go:356] "Generic (PLEG): container finished" podID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerID="1327b73e6bf93d156e322b917e5eb4f9979fa36a2874d0c338fd27db7e14fc42" exitCode=7 Jul 15 13:00:01.933333 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:01.932963 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerDied","Data":"1327b73e6bf93d156e322b917e5eb4f9979fa36a2874d0c338fd27db7e14fc42"} Jul 15 13:00:01.933333 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:01.933096 2559 scope.go:117] "RemoveContainer" containerID="1327b73e6bf93d156e322b917e5eb4f9979fa36a2874d0c338fd27db7e14fc42" Jul 15 13:00:01.934572 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:01.934550 2559 generic.go:356] "Generic (PLEG): container finished" podID="0af0c3e4-6f23-4c81-97eb-7b0847ef285c" containerID="a07822fbece1c93a83903cebd6099d966c4d73dd636fca4b05c9b18497dc1ee3" exitCode=0 Jul 15 13:00:01.934685 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:01.934633 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" event={"ID":"0af0c3e4-6f23-4c81-97eb-7b0847ef285c","Type":"ContainerDied","Data":"a07822fbece1c93a83903cebd6099d966c4d73dd636fca4b05c9b18497dc1ee3"} Jul 15 13:00:02.942709 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:02.942678 2559 generic.go:356] "Generic (PLEG): container finished" podID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerID="f6832aab036849a359f396be8e9154cd395e78f3027248f525e94c7a84cfc3f0" exitCode=7 Jul 15 13:00:02.943161 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:02.942772 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerDied","Data":"f6832aab036849a359f396be8e9154cd395e78f3027248f525e94c7a84cfc3f0"} Jul 15 13:00:02.943161 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:02.942872 2559 scope.go:117] "RemoveContainer" containerID="1327b73e6bf93d156e322b917e5eb4f9979fa36a2874d0c338fd27db7e14fc42" Jul 15 13:00:02.943161 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:02.942933 2559 scope.go:117] "RemoveContainer" containerID="f6832aab036849a359f396be8e9154cd395e78f3027248f525e94c7a84cfc3f0" Jul 15 13:00:02.943600 ip-10-0-142-131 kubenswrapper[2559]: E0715 13:00:02.943209 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd_odh-ai-gateway-infra(07b1d27a-f3fd-4adc-bdb9-d0c493008350)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" Jul 15 13:00:03.088628 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:03.088599 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" Jul 15 13:00:03.950504 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:03.950474 2559 scope.go:117] "RemoveContainer" containerID="f6832aab036849a359f396be8e9154cd395e78f3027248f525e94c7a84cfc3f0" Jul 15 13:00:03.950961 ip-10-0-142-131 kubenswrapper[2559]: E0715 13:00:03.950717 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd_odh-ai-gateway-infra(07b1d27a-f3fd-4adc-bdb9-d0c493008350)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" Jul 15 13:00:03.951692 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:03.951674 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" Jul 15 13:00:03.951741 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:03.951715 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn" event={"ID":"0af0c3e4-6f23-4c81-97eb-7b0847ef285c","Type":"ContainerDied","Data":"361cdbb08693d5f75b725627974c202af84918f6108d6515a23bdf88db3a544f"} Jul 15 13:00:03.951786 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:03.951756 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="361cdbb08693d5f75b725627974c202af84918f6108d6515a23bdf88db3a544f" Jul 15 13:00:16.104319 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:16.104289 2559 scope.go:117] "RemoveContainer" containerID="f6832aab036849a359f396be8e9154cd395e78f3027248f525e94c7a84cfc3f0" Jul 15 13:00:17.035420 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:17.035377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerStarted","Data":"e070a203a4702ff499ed57952d0283da0b9b9084fb5de7c2cf47ed96d12ca9cc"} Jul 15 13:00:18.044215 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:18.044183 2559 generic.go:356] "Generic (PLEG): container finished" podID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerID="e070a203a4702ff499ed57952d0283da0b9b9084fb5de7c2cf47ed96d12ca9cc" exitCode=0 Jul 15 13:00:18.044655 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:18.044257 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerDied","Data":"e070a203a4702ff499ed57952d0283da0b9b9084fb5de7c2cf47ed96d12ca9cc"} Jul 15 13:00:18.044655 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:18.044320 2559 scope.go:117] "RemoveContainer" containerID="f6832aab036849a359f396be8e9154cd395e78f3027248f525e94c7a84cfc3f0" Jul 15 13:00:18.069258 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:18.069234 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd"] Jul 15 13:00:19.191134 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:19.191100 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" Jul 15 13:00:20.060563 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:20.060536 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" Jul 15 13:00:20.060742 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:20.060579 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd" event={"ID":"07b1d27a-f3fd-4adc-bdb9-d0c493008350","Type":"ContainerDied","Data":"5e8605d81772ca4f526b1b68d93784430ab4f9af8b3e31127f2b1f75db0968a7"} Jul 15 13:00:20.060742 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:20.060643 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e8605d81772ca4f526b1b68d93784430ab4f9af8b3e31127f2b1f75db0968a7" Jul 15 13:00:20.112772 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:20.112748 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd"] Jul 15 13:00:20.121572 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:20.121547 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-34f439-29735340-czmdd"] Jul 15 13:00:22.110289 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:22.110256 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" path="/var/lib/kubelet/pods/07b1d27a-f3fd-4adc-bdb9-d0c493008350/volumes" Jul 15 13:00:47.048910 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.048834 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv"] Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049277 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af0c3e4-6f23-4c81-97eb-7b0847ef285c" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049289 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af0c3e4-6f23-4c81-97eb-7b0847ef285c" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049315 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049321 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049327 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049334 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049361 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049365 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049367 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049617 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049440 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049617 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049450 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.049617 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049458 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af0c3e4-6f23-4c81-97eb-7b0847ef285c" containerName="cleanup" Jul 15 13:00:47.049617 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.049464 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b1d27a-f3fd-4adc-bdb9-d0c493008350" containerName="cleanup" Jul 15 13:00:47.052715 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.052700 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:47.056968 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.056950 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 13:00:47.068113 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.068087 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv"] Jul 15 13:00:47.131700 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.131659 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4f28ca-b00e-479e-a763-24920d55bfc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv\" (UID: \"4e4f28ca-b00e-479e-a763-24920d55bfc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:47.232793 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.232751 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4f28ca-b00e-479e-a763-24920d55bfc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv\" (UID: \"4e4f28ca-b00e-479e-a763-24920d55bfc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:47.233450 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.233428 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4f28ca-b00e-479e-a763-24920d55bfc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv\" (UID: \"4e4f28ca-b00e-479e-a763-24920d55bfc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:47.362768 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.362663 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:47.725824 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:47.725783 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv"] Jul 15 13:00:47.727008 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:00:47.726969 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4f28ca_b00e_479e_a763_24920d55bfc6.slice/crio-f74ac19e920902d6df2ecab9b74d338ab2c4e8e6d2476f668e9b4801aab875c5 WatchSource:0}: Error finding container f74ac19e920902d6df2ecab9b74d338ab2c4e8e6d2476f668e9b4801aab875c5: Status 404 returned error can't find the container with id f74ac19e920902d6df2ecab9b74d338ab2c4e8e6d2476f668e9b4801aab875c5 Jul 15 13:00:48.229551 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:48.229459 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" event={"ID":"4e4f28ca-b00e-479e-a763-24920d55bfc6","Type":"ContainerStarted","Data":"7ba4f00a29eca3018a28a05f4ff1d369d2b5a5d5292c0f548b857a72cb73e660"} Jul 15 13:00:48.229551 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:48.229512 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" event={"ID":"4e4f28ca-b00e-479e-a763-24920d55bfc6","Type":"ContainerStarted","Data":"f74ac19e920902d6df2ecab9b74d338ab2c4e8e6d2476f668e9b4801aab875c5"} Jul 15 13:00:48.254776 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:48.254729 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" podStartSLOduration=1.254715231 podStartE2EDuration="1.254715231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:00:48.250952319 +0000 UTC m=+2248.801711853" watchObservedRunningTime="2026-07-15 13:00:48.254715231 +0000 UTC m=+2248.805474762" Jul 15 13:00:49.237363 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:49.237317 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e4f28ca-b00e-479e-a763-24920d55bfc6" containerID="7ba4f00a29eca3018a28a05f4ff1d369d2b5a5d5292c0f548b857a72cb73e660" exitCode=0 Jul 15 13:00:49.237742 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:49.237410 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" event={"ID":"4e4f28ca-b00e-479e-a763-24920d55bfc6","Type":"ContainerDied","Data":"7ba4f00a29eca3018a28a05f4ff1d369d2b5a5d5292c0f548b857a72cb73e660"} Jul 15 13:00:50.382482 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:50.382454 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:50.463142 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:50.463099 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4f28ca-b00e-479e-a763-24920d55bfc6-maas-api-service-ca\") pod \"4e4f28ca-b00e-479e-a763-24920d55bfc6\" (UID: \"4e4f28ca-b00e-479e-a763-24920d55bfc6\") " Jul 15 13:00:50.463511 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:50.463483 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4f28ca-b00e-479e-a763-24920d55bfc6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e4f28ca-b00e-479e-a763-24920d55bfc6" (UID: "4e4f28ca-b00e-479e-a763-24920d55bfc6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:00:50.564705 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:50.564610 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4f28ca-b00e-479e-a763-24920d55bfc6-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:00:51.250551 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:51.250521 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" Jul 15 13:00:51.250722 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:51.250565 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv" event={"ID":"4e4f28ca-b00e-479e-a763-24920d55bfc6","Type":"ContainerDied","Data":"f74ac19e920902d6df2ecab9b74d338ab2c4e8e6d2476f668e9b4801aab875c5"} Jul 15 13:00:51.250722 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:51.250607 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f74ac19e920902d6df2ecab9b74d338ab2c4e8e6d2476f668e9b4801aab875c5" Jul 15 13:00:57.827971 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:57.827941 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn"] Jul 15 13:00:57.835422 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:57.835398 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1fe3a4-29735340-jgvgn"] Jul 15 13:00:58.110034 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:00:58.109942 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af0c3e4-6f23-4c81-97eb-7b0847ef285c" path="/var/lib/kubelet/pods/0af0c3e4-6f23-4c81-97eb-7b0847ef285c/volumes" Jul 15 13:01:13.349662 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.349622 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs"] Jul 15 13:01:13.350147 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.350095 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e4f28ca-b00e-479e-a763-24920d55bfc6" containerName="revoke-keys" Jul 15 13:01:13.350147 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.350108 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e4f28ca-b00e-479e-a763-24920d55bfc6" containerName="revoke-keys" Jul 15 13:01:13.350261 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.350174 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e4f28ca-b00e-479e-a763-24920d55bfc6" containerName="revoke-keys" Jul 15 13:01:13.353724 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.353704 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:13.357561 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.357540 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 13:01:13.365235 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.365208 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs"] Jul 15 13:01:13.483261 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.483231 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8547ec-fea0-4bd1-a1e3-b309d1b262a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-34f439-269cs\" (UID: \"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:13.583955 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.583920 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8547ec-fea0-4bd1-a1e3-b309d1b262a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-34f439-269cs\" (UID: \"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:13.584563 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.584544 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8547ec-fea0-4bd1-a1e3-b309d1b262a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-34f439-269cs\" (UID: \"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:13.666103 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.666013 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:13.813560 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:13.813531 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs"] Jul 15 13:01:13.814444 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:01:13.814412 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8547ec_fea0_4bd1_a1e3_b309d1b262a8.slice/crio-15c7183740b397adc84bff0867ba3ec4664052affab8cdffedcfa7e89d8006d3 WatchSource:0}: Error finding container 15c7183740b397adc84bff0867ba3ec4664052affab8cdffedcfa7e89d8006d3: Status 404 returned error can't find the container with id 15c7183740b397adc84bff0867ba3ec4664052affab8cdffedcfa7e89d8006d3 Jul 15 13:01:14.398670 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:14.398568 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" event={"ID":"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8","Type":"ContainerStarted","Data":"255ffdbbe02ed4dce5ff8555a838c0afe7126d6f60fddca1cf86997015d75582"} Jul 15 13:01:14.398670 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:14.398641 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" event={"ID":"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8","Type":"ContainerStarted","Data":"15c7183740b397adc84bff0867ba3ec4664052affab8cdffedcfa7e89d8006d3"} Jul 15 13:01:14.418064 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:14.418019 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" podStartSLOduration=1.418003348 podStartE2EDuration="1.418003348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:01:14.415172627 +0000 UTC m=+2274.965932181" watchObservedRunningTime="2026-07-15 13:01:14.418003348 +0000 UTC m=+2274.968762879" Jul 15 13:01:15.412602 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:15.412564 2559 generic.go:356] "Generic (PLEG): container finished" podID="4c8547ec-fea0-4bd1-a1e3-b309d1b262a8" containerID="255ffdbbe02ed4dce5ff8555a838c0afe7126d6f60fddca1cf86997015d75582" exitCode=0 Jul 15 13:01:15.413057 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:15.412621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" event={"ID":"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8","Type":"ContainerDied","Data":"255ffdbbe02ed4dce5ff8555a838c0afe7126d6f60fddca1cf86997015d75582"} Jul 15 13:01:16.562490 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:16.562468 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:16.711167 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:16.711137 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8547ec-fea0-4bd1-a1e3-b309d1b262a8-maas-api-service-ca\") pod \"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8\" (UID: \"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8\") " Jul 15 13:01:16.711536 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:16.711509 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c8547ec-fea0-4bd1-a1e3-b309d1b262a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c8547ec-fea0-4bd1-a1e3-b309d1b262a8" (UID: "4c8547ec-fea0-4bd1-a1e3-b309d1b262a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:01:16.811896 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:16.811862 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8547ec-fea0-4bd1-a1e3-b309d1b262a8-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:01:17.426392 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:17.426363 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" Jul 15 13:01:17.426596 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:17.426412 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-34f439-269cs" event={"ID":"4c8547ec-fea0-4bd1-a1e3-b309d1b262a8","Type":"ContainerDied","Data":"15c7183740b397adc84bff0867ba3ec4664052affab8cdffedcfa7e89d8006d3"} Jul 15 13:01:17.426596 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:01:17.426457 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15c7183740b397adc84bff0867ba3ec4664052affab8cdffedcfa7e89d8006d3" Jul 15 13:02:04.031690 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.031654 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t"] Jul 15 13:02:04.032196 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.032133 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c8547ec-fea0-4bd1-a1e3-b309d1b262a8" containerName="revoke-keys" Jul 15 13:02:04.032196 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.032146 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c8547ec-fea0-4bd1-a1e3-b309d1b262a8" containerName="revoke-keys" Jul 15 13:02:04.032281 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.032217 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c8547ec-fea0-4bd1-a1e3-b309d1b262a8" containerName="revoke-keys" Jul 15 13:02:04.035422 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.035403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:04.038209 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.038191 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 13:02:04.046787 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.046764 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t"] Jul 15 13:02:04.156103 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.156076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd0afd1f-6b13-4622-b1ab-d8b66984763f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t\" (UID: \"cd0afd1f-6b13-4622-b1ab-d8b66984763f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:04.257512 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.257487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd0afd1f-6b13-4622-b1ab-d8b66984763f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t\" (UID: \"cd0afd1f-6b13-4622-b1ab-d8b66984763f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:04.258129 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.258110 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd0afd1f-6b13-4622-b1ab-d8b66984763f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t\" (UID: \"cd0afd1f-6b13-4622-b1ab-d8b66984763f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:04.346527 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.346466 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:04.486548 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.486490 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t"] Jul 15 13:02:04.487208 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:02:04.487180 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0afd1f_6b13_4622_b1ab_d8b66984763f.slice/crio-9a1f59a7dc5381c84bdf842afc9909d8cfcdd067ae6d2386ee142f92fc202b97 WatchSource:0}: Error finding container 9a1f59a7dc5381c84bdf842afc9909d8cfcdd067ae6d2386ee142f92fc202b97: Status 404 returned error can't find the container with id 9a1f59a7dc5381c84bdf842afc9909d8cfcdd067ae6d2386ee142f92fc202b97 Jul 15 13:02:04.733218 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:04.733189 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" event={"ID":"cd0afd1f-6b13-4622-b1ab-d8b66984763f","Type":"ContainerStarted","Data":"9a1f59a7dc5381c84bdf842afc9909d8cfcdd067ae6d2386ee142f92fc202b97"} Jul 15 13:02:05.740574 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:05.740537 2559 generic.go:356] "Generic (PLEG): container finished" podID="cd0afd1f-6b13-4622-b1ab-d8b66984763f" containerID="2ec5ebc73306b545afca99abca799ff85f1d2048b26959afc5fcce3c0aee1e8c" exitCode=0 Jul 15 13:02:05.741007 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:05.740627 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" event={"ID":"cd0afd1f-6b13-4622-b1ab-d8b66984763f","Type":"ContainerDied","Data":"2ec5ebc73306b545afca99abca799ff85f1d2048b26959afc5fcce3c0aee1e8c"} Jul 15 13:02:06.889581 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:06.889558 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:07.082172 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:07.082101 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd0afd1f-6b13-4622-b1ab-d8b66984763f-maas-api-service-ca\") pod \"cd0afd1f-6b13-4622-b1ab-d8b66984763f\" (UID: \"cd0afd1f-6b13-4622-b1ab-d8b66984763f\") " Jul 15 13:02:07.082420 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:07.082399 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd0afd1f-6b13-4622-b1ab-d8b66984763f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd0afd1f-6b13-4622-b1ab-d8b66984763f" (UID: "cd0afd1f-6b13-4622-b1ab-d8b66984763f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:02:07.183408 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:07.183375 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd0afd1f-6b13-4622-b1ab-d8b66984763f-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:02:07.753328 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:07.753300 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" Jul 15 13:02:07.753328 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:07.753317 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t" event={"ID":"cd0afd1f-6b13-4622-b1ab-d8b66984763f","Type":"ContainerDied","Data":"9a1f59a7dc5381c84bdf842afc9909d8cfcdd067ae6d2386ee142f92fc202b97"} Jul 15 13:02:07.753556 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:07.753379 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a1f59a7dc5381c84bdf842afc9909d8cfcdd067ae6d2386ee142f92fc202b97" Jul 15 13:02:30.206731 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.206688 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84"] Jul 15 13:02:30.207129 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.207113 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd0afd1f-6b13-4622-b1ab-d8b66984763f" containerName="revoke-keys" Jul 15 13:02:30.207129 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.207125 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd0afd1f-6b13-4622-b1ab-d8b66984763f" containerName="revoke-keys" Jul 15 13:02:30.207224 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.207214 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd0afd1f-6b13-4622-b1ab-d8b66984763f" containerName="revoke-keys" Jul 15 13:02:30.210446 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.210427 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:30.213472 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.213454 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 13:02:30.220303 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.220285 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84"] Jul 15 13:02:30.271172 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.271140 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a2eab84-2647-40a4-963b-23364331094c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8a1d21-xns84\" (UID: \"4a2eab84-2647-40a4-963b-23364331094c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:30.371542 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.371508 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a2eab84-2647-40a4-963b-23364331094c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8a1d21-xns84\" (UID: \"4a2eab84-2647-40a4-963b-23364331094c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:30.372117 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.372099 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a2eab84-2647-40a4-963b-23364331094c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8a1d21-xns84\" (UID: \"4a2eab84-2647-40a4-963b-23364331094c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:30.521370 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.521343 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:30.665850 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.665760 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84"] Jul 15 13:02:30.665963 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:02:30.665866 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2eab84_2647_40a4_963b_23364331094c.slice/crio-0cf779811f62ed9baaef96f2c472c461b69fb4c8f28341ed789eceb97756938e WatchSource:0}: Error finding container 0cf779811f62ed9baaef96f2c472c461b69fb4c8f28341ed789eceb97756938e: Status 404 returned error can't find the container with id 0cf779811f62ed9baaef96f2c472c461b69fb4c8f28341ed789eceb97756938e Jul 15 13:02:30.897959 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:30.897930 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" event={"ID":"4a2eab84-2647-40a4-963b-23364331094c","Type":"ContainerStarted","Data":"0cf779811f62ed9baaef96f2c472c461b69fb4c8f28341ed789eceb97756938e"} Jul 15 13:02:31.905146 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:31.905065 2559 generic.go:356] "Generic (PLEG): container finished" podID="4a2eab84-2647-40a4-963b-23364331094c" containerID="842ef6aef1f3d59f6c8e8d3c294d65b0df561bced40c9095055f95ae09213863" exitCode=0 Jul 15 13:02:31.905500 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:31.905151 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" event={"ID":"4a2eab84-2647-40a4-963b-23364331094c","Type":"ContainerDied","Data":"842ef6aef1f3d59f6c8e8d3c294d65b0df561bced40c9095055f95ae09213863"} Jul 15 13:02:33.050666 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.050642 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:33.095022 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.094993 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a2eab84-2647-40a4-963b-23364331094c-maas-api-service-ca\") pod \"4a2eab84-2647-40a4-963b-23364331094c\" (UID: \"4a2eab84-2647-40a4-963b-23364331094c\") " Jul 15 13:02:33.097882 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.095838 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a2eab84-2647-40a4-963b-23364331094c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a2eab84-2647-40a4-963b-23364331094c" (UID: "4a2eab84-2647-40a4-963b-23364331094c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:02:33.196277 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.196198 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a2eab84-2647-40a4-963b-23364331094c-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:02:33.919770 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.919741 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" Jul 15 13:02:33.919973 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.919776 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8a1d21-xns84" event={"ID":"4a2eab84-2647-40a4-963b-23364331094c","Type":"ContainerDied","Data":"0cf779811f62ed9baaef96f2c472c461b69fb4c8f28341ed789eceb97756938e"} Jul 15 13:02:33.919973 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:02:33.919835 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cf779811f62ed9baaef96f2c472c461b69fb4c8f28341ed789eceb97756938e" Jul 15 13:03:20.194116 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:03:20.194026 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 13:03:20.204664 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:03:20.204645 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 13:06:20.318343 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:06:20.318312 2559 scope.go:117] "RemoveContainer" containerID="e070a203a4702ff499ed57952d0283da0b9b9084fb5de7c2cf47ed96d12ca9cc" Jul 15 13:06:20.327420 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:06:20.327400 2559 scope.go:117] "RemoveContainer" containerID="a07822fbece1c93a83903cebd6099d966c4d73dd636fca4b05c9b18497dc1ee3" Jul 15 13:08:20.244654 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:08:20.244629 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 13:08:20.253310 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:08:20.253284 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 13:09:51.178151 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.178064 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7482-139776711616048"] Jul 15 13:09:51.178563 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.178511 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2eab84-2647-40a4-963b-23364331094c" containerName="revoke-keys" Jul 15 13:09:51.178563 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.178522 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2eab84-2647-40a4-963b-23364331094c" containerName="revoke-keys" Jul 15 13:09:51.178638 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.178598 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a2eab84-2647-40a4-963b-23364331094c" containerName="revoke-keys" Jul 15 13:09:51.181931 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.181915 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:51.184902 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.184867 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-92p4s\"" Jul 15 13:09:51.187752 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.187728 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711616048"] Jul 15 13:09:51.317208 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.317175 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9f67\" (UniqueName: \"kubernetes.io/projected/aace3288-749f-436d-bc3e-c08b191adfa7-kube-api-access-f9f67\") pod \"test-curl-7482-139776711616048\" (UID: \"aace3288-749f-436d-bc3e-c08b191adfa7\") " pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:51.418721 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.418686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9f67\" (UniqueName: \"kubernetes.io/projected/aace3288-749f-436d-bc3e-c08b191adfa7-kube-api-access-f9f67\") pod \"test-curl-7482-139776711616048\" (UID: \"aace3288-749f-436d-bc3e-c08b191adfa7\") " pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:51.427912 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.427879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9f67\" (UniqueName: \"kubernetes.io/projected/aace3288-749f-436d-bc3e-c08b191adfa7-kube-api-access-f9f67\") pod \"test-curl-7482-139776711616048\" (UID: \"aace3288-749f-436d-bc3e-c08b191adfa7\") " pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:51.493931 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.493903 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:51.641088 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.641061 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711616048"] Jul 15 13:09:51.642247 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:09:51.642221 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaace3288_749f_436d_bc3e_c08b191adfa7.slice/crio-7d69e1f82aaf0a1c4991a1d5c4f338bcd15e79a4bcb3de3bb58733a9d40f8d4a WatchSource:0}: Error finding container 7d69e1f82aaf0a1c4991a1d5c4f338bcd15e79a4bcb3de3bb58733a9d40f8d4a: Status 404 returned error can't find the container with id 7d69e1f82aaf0a1c4991a1d5c4f338bcd15e79a4bcb3de3bb58733a9d40f8d4a Jul 15 13:09:51.643618 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.643602 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 13:09:51.664018 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:51.663989 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711616048" event={"ID":"aace3288-749f-436d-bc3e-c08b191adfa7","Type":"ContainerStarted","Data":"7d69e1f82aaf0a1c4991a1d5c4f338bcd15e79a4bcb3de3bb58733a9d40f8d4a"} Jul 15 13:09:52.672435 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:52.672343 2559 generic.go:356] "Generic (PLEG): container finished" podID="aace3288-749f-436d-bc3e-c08b191adfa7" containerID="58206a3350975edc815f74de2c9e1472c6fffae0e74806183c641f604f81b28a" exitCode=0 Jul 15 13:09:52.672919 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:52.672428 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711616048" event={"ID":"aace3288-749f-436d-bc3e-c08b191adfa7","Type":"ContainerDied","Data":"58206a3350975edc815f74de2c9e1472c6fffae0e74806183c641f604f81b28a"} Jul 15 13:09:53.822889 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.822863 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:53.841069 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.841037 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9f67\" (UniqueName: \"kubernetes.io/projected/aace3288-749f-436d-bc3e-c08b191adfa7-kube-api-access-f9f67\") pod \"aace3288-749f-436d-bc3e-c08b191adfa7\" (UID: \"aace3288-749f-436d-bc3e-c08b191adfa7\") " Jul 15 13:09:53.843358 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.843328 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aace3288-749f-436d-bc3e-c08b191adfa7-kube-api-access-f9f67" (OuterVolumeSpecName: "kube-api-access-f9f67") pod "aace3288-749f-436d-bc3e-c08b191adfa7" (UID: "aace3288-749f-436d-bc3e-c08b191adfa7"). InnerVolumeSpecName "kube-api-access-f9f67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:09:53.860772 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.860748 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7482-139776711616048_aace3288-749f-436d-bc3e-c08b191adfa7/test-curl-7482-139776711616048/0.log" Jul 15 13:09:53.891593 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.891559 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7482-139776711616048"] Jul 15 13:09:53.894537 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.894516 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7482-139776711616048"] Jul 15 13:09:53.942275 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:53.942197 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9f67\" (UniqueName: \"kubernetes.io/projected/aace3288-749f-436d-bc3e-c08b191adfa7-kube-api-access-f9f67\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:09:54.008028 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.007996 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7482-139776711615792"] Jul 15 13:09:54.008490 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.008476 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aace3288-749f-436d-bc3e-c08b191adfa7" containerName="test-curl-7482-139776711616048" Jul 15 13:09:54.008538 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.008492 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aace3288-749f-436d-bc3e-c08b191adfa7" containerName="test-curl-7482-139776711616048" Jul 15 13:09:54.008572 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.008563 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aace3288-749f-436d-bc3e-c08b191adfa7" containerName="test-curl-7482-139776711616048" Jul 15 13:09:54.011788 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.011771 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:54.018923 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.018897 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711615792"] Jul 15 13:09:54.042823 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.042771 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm75j\" (UniqueName: \"kubernetes.io/projected/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a-kube-api-access-gm75j\") pod \"test-curl-7482-139776711615792\" (UID: \"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a\") " pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:54.111050 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.111009 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aace3288-749f-436d-bc3e-c08b191adfa7" path="/var/lib/kubelet/pods/aace3288-749f-436d-bc3e-c08b191adfa7/volumes" Jul 15 13:09:54.143891 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.143852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm75j\" (UniqueName: \"kubernetes.io/projected/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a-kube-api-access-gm75j\") pod \"test-curl-7482-139776711615792\" (UID: \"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a\") " pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:54.153219 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.153194 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm75j\" (UniqueName: \"kubernetes.io/projected/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a-kube-api-access-gm75j\") pod \"test-curl-7482-139776711615792\" (UID: \"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a\") " pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:54.322721 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.322685 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:54.673701 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:09:54.673664 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2d85faf_ed3a_4d2d_9466_22d4a05e8c6a.slice/crio-b727bc53072318249c29dffdc5be5cfa7123e4efc0508761fabd007b63aa378b WatchSource:0}: Error finding container b727bc53072318249c29dffdc5be5cfa7123e4efc0508761fabd007b63aa378b: Status 404 returned error can't find the container with id b727bc53072318249c29dffdc5be5cfa7123e4efc0508761fabd007b63aa378b Jul 15 13:09:54.674318 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.674283 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711615792"] Jul 15 13:09:54.685609 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.685584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711615792" event={"ID":"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a","Type":"ContainerStarted","Data":"b727bc53072318249c29dffdc5be5cfa7123e4efc0508761fabd007b63aa378b"} Jul 15 13:09:54.686988 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.686969 2559 scope.go:117] "RemoveContainer" containerID="58206a3350975edc815f74de2c9e1472c6fffae0e74806183c641f604f81b28a" Jul 15 13:09:54.687094 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:54.686972 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711616048" Jul 15 13:09:55.695278 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:55.695242 2559 generic.go:356] "Generic (PLEG): container finished" podID="a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" containerID="c9b161d97848a3ed4ffd4f21470064cf466687a09e723aad7a995d8d5a4eaed1" exitCode=0 Jul 15 13:09:55.695278 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:55.695280 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711615792" event={"ID":"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a","Type":"ContainerDied","Data":"c9b161d97848a3ed4ffd4f21470064cf466687a09e723aad7a995d8d5a4eaed1"} Jul 15 13:09:56.844538 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.844510 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:56.866375 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.866344 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7482-139776711615792_a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a/test-curl-7482-139776711615792/0.log" Jul 15 13:09:56.866740 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.866720 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm75j\" (UniqueName: \"kubernetes.io/projected/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a-kube-api-access-gm75j\") pod \"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a\" (UID: \"a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a\") " Jul 15 13:09:56.871151 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.871123 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a-kube-api-access-gm75j" (OuterVolumeSpecName: "kube-api-access-gm75j") pod "a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" (UID: "a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a"). InnerVolumeSpecName "kube-api-access-gm75j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:09:56.891522 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.891489 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7482-139776711615792"] Jul 15 13:09:56.896400 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.896374 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7482-139776711615792"] Jul 15 13:09:56.967542 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:56.967515 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm75j\" (UniqueName: \"kubernetes.io/projected/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a-kube-api-access-gm75j\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:09:57.051184 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.051151 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:09:57.051612 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.051599 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" containerName="test-curl-7482-139776711615792" Jul 15 13:09:57.051660 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.051613 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" containerName="test-curl-7482-139776711615792" Jul 15 13:09:57.051705 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.051696 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" containerName="test-curl-7482-139776711615792" Jul 15 13:09:57.054874 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.054855 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:09:57.062661 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.062637 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:09:57.068108 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.068084 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt94n\" (UniqueName: \"kubernetes.io/projected/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0-kube-api-access-mt94n\") pod \"test-curl-7482-139776711423728\" (UID: \"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0\") " pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:09:57.168735 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.168690 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt94n\" (UniqueName: \"kubernetes.io/projected/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0-kube-api-access-mt94n\") pod \"test-curl-7482-139776711423728\" (UID: \"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0\") " pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:09:57.177752 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.177721 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt94n\" (UniqueName: \"kubernetes.io/projected/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0-kube-api-access-mt94n\") pod \"test-curl-7482-139776711423728\" (UID: \"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0\") " pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:09:57.365195 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.365096 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:09:57.512484 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.512455 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:09:57.512910 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:09:57.512888 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67b2a162_26a9_4d5d_af6d_f3d0c2606ea0.slice/crio-8aeea3a54daf4de16a6fc38e5d345bd1082111c7e5ea9f87fee9ab924c0dce09 WatchSource:0}: Error finding container 8aeea3a54daf4de16a6fc38e5d345bd1082111c7e5ea9f87fee9ab924c0dce09: Status 404 returned error can't find the container with id 8aeea3a54daf4de16a6fc38e5d345bd1082111c7e5ea9f87fee9ab924c0dce09 Jul 15 13:09:57.709295 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.709273 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:57.709468 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.709296 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b727bc53072318249c29dffdc5be5cfa7123e4efc0508761fabd007b63aa378b" Jul 15 13:09:57.710587 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:57.710564 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711423728" event={"ID":"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0","Type":"ContainerStarted","Data":"8aeea3a54daf4de16a6fc38e5d345bd1082111c7e5ea9f87fee9ab924c0dce09"} Jul 15 13:09:57.720220 ip-10-0-142-131 kubenswrapper[2559]: E0715 13:09:57.720197 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7482-139776711615792\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" pod="opendatahub/test-curl-7482-139776711615792" Jul 15 13:09:58.110908 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:58.110826 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a" path="/var/lib/kubelet/pods/a2d85faf-ed3a-4d2d-9466-22d4a05e8c6a/volumes" Jul 15 13:09:58.718321 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:58.718286 2559 generic.go:356] "Generic (PLEG): container finished" podID="67b2a162-26a9-4d5d-af6d-f3d0c2606ea0" containerID="91ccb38fbe6deb0adb44bc58adaaa27964fcb912db52d35f44be914288457d24" exitCode=0 Jul 15 13:09:58.718512 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:58.718378 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711423728" event={"ID":"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0","Type":"ContainerDied","Data":"91ccb38fbe6deb0adb44bc58adaaa27964fcb912db52d35f44be914288457d24"} Jul 15 13:09:59.862922 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.862896 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:09:59.886041 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.886011 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7482-139776711423728_67b2a162-26a9-4d5d-af6d-f3d0c2606ea0/test-curl-7482-139776711423728/0.log" Jul 15 13:09:59.892216 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.892186 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt94n\" (UniqueName: \"kubernetes.io/projected/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0-kube-api-access-mt94n\") pod \"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0\" (UID: \"67b2a162-26a9-4d5d-af6d-f3d0c2606ea0\") " Jul 15 13:09:59.894327 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.894297 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0-kube-api-access-mt94n" (OuterVolumeSpecName: "kube-api-access-mt94n") pod "67b2a162-26a9-4d5d-af6d-f3d0c2606ea0" (UID: "67b2a162-26a9-4d5d-af6d-f3d0c2606ea0"). InnerVolumeSpecName "kube-api-access-mt94n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:09:59.912395 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.912365 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:09:59.915152 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.915131 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:09:59.994024 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:09:59.993937 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt94n\" (UniqueName: \"kubernetes.io/projected/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0-kube-api-access-mt94n\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:10:00.029584 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.029549 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:10:00.030090 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.030076 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67b2a162-26a9-4d5d-af6d-f3d0c2606ea0" containerName="test-curl-7482-139776711423728" Jul 15 13:10:00.030136 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.030092 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67b2a162-26a9-4d5d-af6d-f3d0c2606ea0" containerName="test-curl-7482-139776711423728" Jul 15 13:10:00.030194 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.030184 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67b2a162-26a9-4d5d-af6d-f3d0c2606ea0" containerName="test-curl-7482-139776711423728" Jul 15 13:10:00.033291 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.033275 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:00.042633 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.042605 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:10:00.095333 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.095293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8f6j\" (UniqueName: \"kubernetes.io/projected/035805b1-ee2a-4e02-b981-adb92b6703da-kube-api-access-t8f6j\") pod \"test-curl-7482-139776711423728\" (UID: \"035805b1-ee2a-4e02-b981-adb92b6703da\") " pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:00.110959 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.110924 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67b2a162-26a9-4d5d-af6d-f3d0c2606ea0" path="/var/lib/kubelet/pods/67b2a162-26a9-4d5d-af6d-f3d0c2606ea0/volumes" Jul 15 13:10:00.196130 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.196093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8f6j\" (UniqueName: \"kubernetes.io/projected/035805b1-ee2a-4e02-b981-adb92b6703da-kube-api-access-t8f6j\") pod \"test-curl-7482-139776711423728\" (UID: \"035805b1-ee2a-4e02-b981-adb92b6703da\") " pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:00.205241 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.205214 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8f6j\" (UniqueName: \"kubernetes.io/projected/035805b1-ee2a-4e02-b981-adb92b6703da-kube-api-access-t8f6j\") pod \"test-curl-7482-139776711423728\" (UID: \"035805b1-ee2a-4e02-b981-adb92b6703da\") " pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:00.343466 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.343376 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:00.489455 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.489426 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:10:00.490362 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:10:00.490336 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod035805b1_ee2a_4e02_b981_adb92b6703da.slice/crio-1654500a901dc3359f77dabc9548036e28ca910ed8881729ed45cadf48a48057 WatchSource:0}: Error finding container 1654500a901dc3359f77dabc9548036e28ca910ed8881729ed45cadf48a48057: Status 404 returned error can't find the container with id 1654500a901dc3359f77dabc9548036e28ca910ed8881729ed45cadf48a48057 Jul 15 13:10:00.732231 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.732201 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:00.732231 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.732222 2559 scope.go:117] "RemoveContainer" containerID="91ccb38fbe6deb0adb44bc58adaaa27964fcb912db52d35f44be914288457d24" Jul 15 13:10:00.733595 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:00.733515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711423728" event={"ID":"035805b1-ee2a-4e02-b981-adb92b6703da","Type":"ContainerStarted","Data":"1654500a901dc3359f77dabc9548036e28ca910ed8881729ed45cadf48a48057"} Jul 15 13:10:01.742351 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:01.742314 2559 generic.go:356] "Generic (PLEG): container finished" podID="035805b1-ee2a-4e02-b981-adb92b6703da" containerID="df05cf0a28757367176b16c2b4f89c46e6079ea55207b21a0db77013860547e1" exitCode=0 Jul 15 13:10:01.742748 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:01.742394 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776711423728" event={"ID":"035805b1-ee2a-4e02-b981-adb92b6703da","Type":"ContainerDied","Data":"df05cf0a28757367176b16c2b4f89c46e6079ea55207b21a0db77013860547e1"} Jul 15 13:10:02.895706 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:02.895680 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:02.922068 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:02.922015 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8f6j\" (UniqueName: \"kubernetes.io/projected/035805b1-ee2a-4e02-b981-adb92b6703da-kube-api-access-t8f6j\") pod \"035805b1-ee2a-4e02-b981-adb92b6703da\" (UID: \"035805b1-ee2a-4e02-b981-adb92b6703da\") " Jul 15 13:10:02.924136 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:02.924112 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035805b1-ee2a-4e02-b981-adb92b6703da-kube-api-access-t8f6j" (OuterVolumeSpecName: "kube-api-access-t8f6j") pod "035805b1-ee2a-4e02-b981-adb92b6703da" (UID: "035805b1-ee2a-4e02-b981-adb92b6703da"). InnerVolumeSpecName "kube-api-access-t8f6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:10:02.936303 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:02.936270 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7482-139776711423728_035805b1-ee2a-4e02-b981-adb92b6703da/test-curl-7482-139776711423728/0.log" Jul 15 13:10:02.976480 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:02.976378 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:10:02.983185 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:02.983155 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7482-139776711423728"] Jul 15 13:10:03.023661 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.023561 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8f6j\" (UniqueName: \"kubernetes.io/projected/035805b1-ee2a-4e02-b981-adb92b6703da-kube-api-access-t8f6j\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:10:03.101272 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.101230 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:03.101733 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.101718 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="035805b1-ee2a-4e02-b981-adb92b6703da" containerName="test-curl-7482-139776711423728" Jul 15 13:10:03.101781 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.101734 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="035805b1-ee2a-4e02-b981-adb92b6703da" containerName="test-curl-7482-139776711423728" Jul 15 13:10:03.101866 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.101856 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="035805b1-ee2a-4e02-b981-adb92b6703da" containerName="test-curl-7482-139776711423728" Jul 15 13:10:03.105175 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.105152 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:03.116004 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.115974 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:03.124993 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.124961 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppqv9\" (UniqueName: \"kubernetes.io/projected/30f6beb3-33b5-4b81-a351-260b04f610ef-kube-api-access-ppqv9\") pod \"test-curl-7482-139776712463680\" (UID: \"30f6beb3-33b5-4b81-a351-260b04f610ef\") " pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:03.225947 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.225905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppqv9\" (UniqueName: \"kubernetes.io/projected/30f6beb3-33b5-4b81-a351-260b04f610ef-kube-api-access-ppqv9\") pod \"test-curl-7482-139776712463680\" (UID: \"30f6beb3-33b5-4b81-a351-260b04f610ef\") " pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:03.235455 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.235428 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppqv9\" (UniqueName: \"kubernetes.io/projected/30f6beb3-33b5-4b81-a351-260b04f610ef-kube-api-access-ppqv9\") pod \"test-curl-7482-139776712463680\" (UID: \"30f6beb3-33b5-4b81-a351-260b04f610ef\") " pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:03.416520 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.416434 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:03.557316 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.557290 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:03.558572 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:10:03.558543 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30f6beb3_33b5_4b81_a351_260b04f610ef.slice/crio-dd127ce1cc23ad7e3dff9c00e6161006321d26f706384ae1f3da9c3760bb5605 WatchSource:0}: Error finding container dd127ce1cc23ad7e3dff9c00e6161006321d26f706384ae1f3da9c3760bb5605: Status 404 returned error can't find the container with id dd127ce1cc23ad7e3dff9c00e6161006321d26f706384ae1f3da9c3760bb5605 Jul 15 13:10:03.755306 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.755269 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776712463680" event={"ID":"30f6beb3-33b5-4b81-a351-260b04f610ef","Type":"ContainerStarted","Data":"dd127ce1cc23ad7e3dff9c00e6161006321d26f706384ae1f3da9c3760bb5605"} Jul 15 13:10:03.756656 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.756634 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1654500a901dc3359f77dabc9548036e28ca910ed8881729ed45cadf48a48057" Jul 15 13:10:03.756656 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:03.756640 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:03.767838 ip-10-0-142-131 kubenswrapper[2559]: E0715 13:10:03.767796 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7482-139776711423728\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="035805b1-ee2a-4e02-b981-adb92b6703da" pod="opendatahub/test-curl-7482-139776711423728" Jul 15 13:10:04.110488 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:04.110454 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="035805b1-ee2a-4e02-b981-adb92b6703da" path="/var/lib/kubelet/pods/035805b1-ee2a-4e02-b981-adb92b6703da/volumes" Jul 15 13:10:04.764498 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:04.764464 2559 generic.go:356] "Generic (PLEG): container finished" podID="30f6beb3-33b5-4b81-a351-260b04f610ef" containerID="cce15eaa97336099f07af6237754907c38a1f4ddaa843256f19c1cd0c5d2bd3c" exitCode=0 Jul 15 13:10:04.764697 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:04.764528 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776712463680" event={"ID":"30f6beb3-33b5-4b81-a351-260b04f610ef","Type":"ContainerDied","Data":"cce15eaa97336099f07af6237754907c38a1f4ddaa843256f19c1cd0c5d2bd3c"} Jul 15 13:10:05.921741 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:05.921712 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:05.952278 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:05.952250 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppqv9\" (UniqueName: \"kubernetes.io/projected/30f6beb3-33b5-4b81-a351-260b04f610ef-kube-api-access-ppqv9\") pod \"30f6beb3-33b5-4b81-a351-260b04f610ef\" (UID: \"30f6beb3-33b5-4b81-a351-260b04f610ef\") " Jul 15 13:10:05.954707 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:05.954651 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f6beb3-33b5-4b81-a351-260b04f610ef-kube-api-access-ppqv9" (OuterVolumeSpecName: "kube-api-access-ppqv9") pod "30f6beb3-33b5-4b81-a351-260b04f610ef" (UID: "30f6beb3-33b5-4b81-a351-260b04f610ef"). InnerVolumeSpecName "kube-api-access-ppqv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:10:05.955847 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:05.955793 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7482-139776712463680_30f6beb3-33b5-4b81-a351-260b04f610ef/test-curl-7482-139776712463680/0.log" Jul 15 13:10:05.982047 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:05.982009 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:05.984908 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:05.984887 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:06.053912 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.053830 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppqv9\" (UniqueName: \"kubernetes.io/projected/30f6beb3-33b5-4b81-a351-260b04f610ef-kube-api-access-ppqv9\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:10:06.111444 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.111405 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30f6beb3-33b5-4b81-a351-260b04f610ef" path="/var/lib/kubelet/pods/30f6beb3-33b5-4b81-a351-260b04f610ef/volumes" Jul 15 13:10:06.112719 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.112691 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:06.113141 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.113122 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f6beb3-33b5-4b81-a351-260b04f610ef" containerName="test-curl-7482-139776712463680" Jul 15 13:10:06.113141 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.113138 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f6beb3-33b5-4b81-a351-260b04f610ef" containerName="test-curl-7482-139776712463680" Jul 15 13:10:06.113288 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.113241 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30f6beb3-33b5-4b81-a351-260b04f610ef" containerName="test-curl-7482-139776712463680" Jul 15 13:10:06.116478 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.116459 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:06.144025 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.143996 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:06.154813 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.154784 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb2c7\" (UniqueName: \"kubernetes.io/projected/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6-kube-api-access-hb2c7\") pod \"test-curl-7482-139776712463680\" (UID: \"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6\") " pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:06.255824 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.255759 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb2c7\" (UniqueName: \"kubernetes.io/projected/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6-kube-api-access-hb2c7\") pod \"test-curl-7482-139776712463680\" (UID: \"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6\") " pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:06.264947 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.264915 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb2c7\" (UniqueName: \"kubernetes.io/projected/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6-kube-api-access-hb2c7\") pod \"test-curl-7482-139776712463680\" (UID: \"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6\") " pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:06.425880 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.425772 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:06.575575 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.575545 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:06.576183 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:10:06.576158 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39709ea9_60a2_4a7f_ae6e_24d6240ca8e6.slice/crio-f243ffcd9d67c1eb2ed53dcb2fde60cec2bb1ceb05095d638dec77bea6277c7e WatchSource:0}: Error finding container f243ffcd9d67c1eb2ed53dcb2fde60cec2bb1ceb05095d638dec77bea6277c7e: Status 404 returned error can't find the container with id f243ffcd9d67c1eb2ed53dcb2fde60cec2bb1ceb05095d638dec77bea6277c7e Jul 15 13:10:06.781431 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.781405 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:06.781637 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.781413 2559 scope.go:117] "RemoveContainer" containerID="cce15eaa97336099f07af6237754907c38a1f4ddaa843256f19c1cd0c5d2bd3c" Jul 15 13:10:06.782862 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:06.782836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776712463680" event={"ID":"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6","Type":"ContainerStarted","Data":"f243ffcd9d67c1eb2ed53dcb2fde60cec2bb1ceb05095d638dec77bea6277c7e"} Jul 15 13:10:07.792665 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:07.792629 2559 generic.go:356] "Generic (PLEG): container finished" podID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" containerID="efbcf35199d18da1f69667cd8f7a98db7f8a60909946abb6b0681da7c71e0fe0" exitCode=0 Jul 15 13:10:07.793112 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:07.792724 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7482-139776712463680" event={"ID":"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6","Type":"ContainerDied","Data":"efbcf35199d18da1f69667cd8f7a98db7f8a60909946abb6b0681da7c71e0fe0"} Jul 15 13:10:08.949363 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:08.949336 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:08.984493 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:08.984461 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb2c7\" (UniqueName: \"kubernetes.io/projected/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6-kube-api-access-hb2c7\") pod \"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6\" (UID: \"39709ea9-60a2-4a7f-ae6e-24d6240ca8e6\") " Jul 15 13:10:08.987501 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:08.987467 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6-kube-api-access-hb2c7" (OuterVolumeSpecName: "kube-api-access-hb2c7") pod "39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" (UID: "39709ea9-60a2-4a7f-ae6e-24d6240ca8e6"). InnerVolumeSpecName "kube-api-access-hb2c7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:10:09.002438 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.002405 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7482-139776712463680_39709ea9-60a2-4a7f-ae6e-24d6240ca8e6/test-curl-7482-139776712463680/0.log" Jul 15 13:10:09.049218 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.049181 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:09.055089 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.055019 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7482-139776712463680"] Jul 15 13:10:09.086249 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.086217 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb2c7\" (UniqueName: \"kubernetes.io/projected/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6-kube-api-access-hb2c7\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:10:09.242911 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.242867 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz"] Jul 15 13:10:09.243387 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.243368 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" containerName="test-curl-7482-139776712463680" Jul 15 13:10:09.243479 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.243390 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" containerName="test-curl-7482-139776712463680" Jul 15 13:10:09.243555 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.243542 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" containerName="test-curl-7482-139776712463680" Jul 15 13:10:09.246924 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.246899 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:09.249617 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.249597 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2gg6\"" Jul 15 13:10:09.258513 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.258467 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz"] Jul 15 13:10:09.289310 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.289265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7d395-0b67-4c57-b07c-352adc124771-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz\" (UID: \"d9f7d395-0b67-4c57-b07c-352adc124771\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:09.390840 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.390718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7d395-0b67-4c57-b07c-352adc124771-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz\" (UID: \"d9f7d395-0b67-4c57-b07c-352adc124771\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:09.391552 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.391525 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7d395-0b67-4c57-b07c-352adc124771-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz\" (UID: \"d9f7d395-0b67-4c57-b07c-352adc124771\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:09.558979 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.558946 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:09.709554 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.709521 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz"] Jul 15 13:10:09.710888 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:10:09.710861 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f7d395_0b67_4c57_b07c_352adc124771.slice/crio-c8b7bccf63fdb449b9ced4707af2ac1b4e9152c0bda73d902990667b7f891e3c WatchSource:0}: Error finding container c8b7bccf63fdb449b9ced4707af2ac1b4e9152c0bda73d902990667b7f891e3c: Status 404 returned error can't find the container with id c8b7bccf63fdb449b9ced4707af2ac1b4e9152c0bda73d902990667b7f891e3c Jul 15 13:10:09.807518 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.807476 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" event={"ID":"d9f7d395-0b67-4c57-b07c-352adc124771","Type":"ContainerStarted","Data":"c8b7bccf63fdb449b9ced4707af2ac1b4e9152c0bda73d902990667b7f891e3c"} Jul 15 13:10:09.808967 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.808942 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:09.809104 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:09.808979 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f243ffcd9d67c1eb2ed53dcb2fde60cec2bb1ceb05095d638dec77bea6277c7e" Jul 15 13:10:09.819759 ip-10-0-142-131 kubenswrapper[2559]: E0715 13:10:09.819729 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7482-139776712463680\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:10.110377 ip-10-0-142-131 kubenswrapper[2559]: E0715 13:10:10.110341 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7482-139776712463680\" is forbidden: User \"system:node:ip-10-0-142-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-131.ec2.internal' and this object" podUID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" pod="opendatahub/test-curl-7482-139776712463680" Jul 15 13:10:10.111350 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:10.111327 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39709ea9-60a2-4a7f-ae6e-24d6240ca8e6" path="/var/lib/kubelet/pods/39709ea9-60a2-4a7f-ae6e-24d6240ca8e6/volumes" Jul 15 13:10:10.816994 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:10.816892 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9f7d395-0b67-4c57-b07c-352adc124771" containerID="63b4000325b029958d292f74f104da54de1ac57dbe5be5bd6fa8943d7215370b" exitCode=0 Jul 15 13:10:10.816994 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:10.816982 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" event={"ID":"d9f7d395-0b67-4c57-b07c-352adc124771","Type":"ContainerDied","Data":"63b4000325b029958d292f74f104da54de1ac57dbe5be5bd6fa8943d7215370b"} Jul 15 13:10:11.965050 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:11.965020 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:12.018859 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:12.018829 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7d395-0b67-4c57-b07c-352adc124771-maas-api-service-ca\") pod \"d9f7d395-0b67-4c57-b07c-352adc124771\" (UID: \"d9f7d395-0b67-4c57-b07c-352adc124771\") " Jul 15 13:10:12.019273 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:12.019246 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9f7d395-0b67-4c57-b07c-352adc124771-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9f7d395-0b67-4c57-b07c-352adc124771" (UID: "d9f7d395-0b67-4c57-b07c-352adc124771"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:10:12.119566 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:12.119540 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7d395-0b67-4c57-b07c-352adc124771-maas-api-service-ca\") on node \"ip-10-0-142-131.ec2.internal\" DevicePath \"\"" Jul 15 13:10:12.831912 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:12.831886 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" Jul 15 13:10:12.832095 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:12.831922 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz" event={"ID":"d9f7d395-0b67-4c57-b07c-352adc124771","Type":"ContainerDied","Data":"c8b7bccf63fdb449b9ced4707af2ac1b4e9152c0bda73d902990667b7f891e3c"} Jul 15 13:10:12.832095 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:10:12.831974 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8b7bccf63fdb449b9ced4707af2ac1b4e9152c0bda73d902990667b7f891e3c" Jul 15 13:11:01.810694 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:01.810662 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c67f569c6-nftv7_0f9af326-4b15-440d-9e94-d3aa668fbfa4/authorino/0.log" Jul 15 13:11:07.197630 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.197596 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tkrr4_d8858c84-c69a-4b01-bdbc-38b065a7508b/manager/0.log" Jul 15 13:11:07.317891 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.317863 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xm9ck_f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466/manager/0.log" Jul 15 13:11:07.438451 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.438424 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fc57dd599-d9rk9_87e69da8-f9ac-45fd-befa-4a707a06ff6b/manager/0.log" Jul 15 13:11:07.560671 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.560645 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vff75_e6cdba19-5aaf-4219-a329-1e733a2391db/server/0.log" Jul 15 13:11:07.676666 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.676640 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cwkbf_ecb74447-252c-4ce1-978b-c1cba7ddc4fa/manager/0.log" Jul 15 13:11:07.794223 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.794195 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-654f68fd79-6zqrk_00e13959-81d2-47d0-962f-d4db59be27da/copy-manifests/0.log" Jul 15 13:11:07.802286 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:07.802265 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-654f68fd79-6zqrk_00e13959-81d2-47d0-962f-d4db59be27da/manager/0.log" Jul 15 13:11:09.002620 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.002595 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-9c73fd-cgr4t_cd0afd1f-6b13-4622-b1ab-d8b66984763f/revoke-keys/0.log" Jul 15 13:11:09.116469 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.116442 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-8a1d21-xns84_4a2eab84-2647-40a4-963b-23364331094c/revoke-keys/0.log" Jul 15 13:11:09.347649 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.347572 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-1fe3a4-zrgrv_4e4f28ca-b00e-479e-a763-24920d55bfc6/revoke-keys/0.log" Jul 15 13:11:09.454952 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.454930 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-34f439-269cs_4c8547ec-fea0-4bd1-a1e3-b309d1b262a8/revoke-keys/0.log" Jul 15 13:11:09.564217 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.564198 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-6532cca1-vmm4m_0c568951-1ab3-41e3-8bc1-50ba509f67b6/revoke-keys/0.log" Jul 15 13:11:09.678999 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.678943 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-8635587b-pmw7z_5c938f35-4879-417b-9e5a-9f7da205e91c/revoke-keys/0.log" Jul 15 13:11:09.789889 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.789867 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-a2ddf8f4-ddldt_e20c08b1-4490-4daf-9833-4dbcf2b3780c/revoke-keys/0.log" Jul 15 13:11:09.897244 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:09.897220 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-ce0265-cbfwz_d9f7d395-0b67-4c57-b07c-352adc124771/revoke-keys/0.log" Jul 15 13:11:10.126497 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:10.126463 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-b92t2_a3faad9f-532f-4943-95be-1a5049cc9024/postgres/0.log" Jul 15 13:11:10.892969 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:10.892936 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76_27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f/util/0.log" Jul 15 13:11:10.899487 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:10.899466 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76_27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f/pull/0.log" Jul 15 13:11:10.906253 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:10.906235 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76_27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f/extract/0.log" Jul 15 13:11:11.015743 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.015723 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2_23c61466-535d-433d-ba92-d9eaf1684a9e/util/0.log" Jul 15 13:11:11.022638 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.022618 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2_23c61466-535d-433d-ba92-d9eaf1684a9e/pull/0.log" Jul 15 13:11:11.029286 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.029268 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2_23c61466-535d-433d-ba92-d9eaf1684a9e/extract/0.log" Jul 15 13:11:11.140920 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.140895 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6_5596d224-0d77-437a-ad3d-971e8c95b127/util/0.log" Jul 15 13:11:11.147835 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.147774 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6_5596d224-0d77-437a-ad3d-971e8c95b127/pull/0.log" Jul 15 13:11:11.154294 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.154278 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6_5596d224-0d77-437a-ad3d-971e8c95b127/extract/0.log" Jul 15 13:11:11.271351 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.271333 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg_914c99bc-2302-4dc1-bf11-028d24766371/util/0.log" Jul 15 13:11:11.280633 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.280614 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg_914c99bc-2302-4dc1-bf11-028d24766371/pull/0.log" Jul 15 13:11:11.287758 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.287741 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg_914c99bc-2302-4dc1-bf11-028d24766371/extract/0.log" Jul 15 13:11:11.406054 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.405996 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c67f569c6-nftv7_0f9af326-4b15-440d-9e94-d3aa668fbfa4/authorino/0.log" Jul 15 13:11:11.536723 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.536700 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-58dzq_9709b468-dc18-488e-8931-3f51e1318abc/manager/0.log" Jul 15 13:11:11.653636 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.653616 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6rbm2_8ff0870c-aa56-4af7-b340-28e9470df6ae/manager/0.log" Jul 15 13:11:11.881860 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:11.881840 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dmhq5_9ec0e4a2-d10d-4e20-adbb-75f805848c47/registry-server/0.log" Jul 15 13:11:12.004517 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:12.004495 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dpgl5_0cc9cc6f-001e-4852-8c91-233785c0d043/manager/0.log" Jul 15 13:11:12.579529 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:12.579496 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fqz8m_40c38105-86fe-4c9b-9c8c-8958a54f7203/istio-proxy/0.log" Jul 15 13:11:12.802713 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:12.802659 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9797f867b-j97px_ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3/kube-auth-proxy/0.log" Jul 15 13:11:13.038030 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:13.038003 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/1.log" Jul 15 13:11:13.152651 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:13.152625 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-xnldf_1b0dabbe-e728-4305-89c1-6425a32e3918/payload-pre-processing/0.log" Jul 15 13:11:13.267187 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:13.267163 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fmfd5_783c2fa1-4ace-4f75-b866-f2f37de9fee9/payload-processing/0.log" Jul 15 13:11:13.381268 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:13.381207 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c7ddc6b-857qm_1effb694-29c1-4b54-8822-604b929a4284/router/0.log" Jul 15 13:11:13.733573 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:13.733546 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dj9h2_30115da7-3fe6-49f8-9dc6-8243232d9fa7/main/0.log" Jul 15 13:11:13.844528 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:13.844503 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-48r7x_1c92dac6-10a1-425b-bcb6-b93882073859/main/0.log" Jul 15 13:11:14.321276 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:14.321250 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vsrrt_fcf8af06-b64a-4219-9d78-6b9398293541/main/0.log" Jul 15 13:11:19.803372 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.803341 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9nr7q/must-gather-wqwxw"] Jul 15 13:11:19.803750 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.803740 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9f7d395-0b67-4c57-b07c-352adc124771" containerName="revoke-keys" Jul 15 13:11:19.803794 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.803752 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9f7d395-0b67-4c57-b07c-352adc124771" containerName="revoke-keys" Jul 15 13:11:19.803872 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.803845 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9f7d395-0b67-4c57-b07c-352adc124771" containerName="revoke-keys" Jul 15 13:11:19.807248 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.807223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:19.812702 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.812682 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9nr7q\"/\"kube-root-ca.crt\"" Jul 15 13:11:19.812839 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.812734 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9nr7q\"/\"default-dockercfg-xwh4g\"" Jul 15 13:11:19.812839 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.812746 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9nr7q\"/\"openshift-service-ca.crt\"" Jul 15 13:11:19.827676 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.827650 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9nr7q/must-gather-wqwxw"] Jul 15 13:11:19.918822 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.918775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfe83427-9989-4b67-b740-30f7f99836a4-must-gather-output\") pod \"must-gather-wqwxw\" (UID: \"cfe83427-9989-4b67-b740-30f7f99836a4\") " pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:19.918989 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:19.918972 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65tlf\" (UniqueName: \"kubernetes.io/projected/cfe83427-9989-4b67-b740-30f7f99836a4-kube-api-access-65tlf\") pod \"must-gather-wqwxw\" (UID: \"cfe83427-9989-4b67-b740-30f7f99836a4\") " pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:20.019754 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.019725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65tlf\" (UniqueName: \"kubernetes.io/projected/cfe83427-9989-4b67-b740-30f7f99836a4-kube-api-access-65tlf\") pod \"must-gather-wqwxw\" (UID: \"cfe83427-9989-4b67-b740-30f7f99836a4\") " pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:20.019946 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.019768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfe83427-9989-4b67-b740-30f7f99836a4-must-gather-output\") pod \"must-gather-wqwxw\" (UID: \"cfe83427-9989-4b67-b740-30f7f99836a4\") " pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:20.020195 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.020174 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfe83427-9989-4b67-b740-30f7f99836a4-must-gather-output\") pod \"must-gather-wqwxw\" (UID: \"cfe83427-9989-4b67-b740-30f7f99836a4\") " pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:20.063780 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.063717 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9nr7q\"/\"kube-root-ca.crt\"" Jul 15 13:11:20.073029 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.073011 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9nr7q\"/\"openshift-service-ca.crt\"" Jul 15 13:11:20.081840 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.081793 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65tlf\" (UniqueName: \"kubernetes.io/projected/cfe83427-9989-4b67-b740-30f7f99836a4-kube-api-access-65tlf\") pod \"must-gather-wqwxw\" (UID: \"cfe83427-9989-4b67-b740-30f7f99836a4\") " pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:20.123901 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.123770 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9nr7q\"/\"default-dockercfg-xwh4g\"" Jul 15 13:11:20.127080 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.127061 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9nr7q/must-gather-wqwxw" Jul 15 13:11:20.268628 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:20.268597 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9nr7q/must-gather-wqwxw"] Jul 15 13:11:20.269723 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:11:20.269697 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfe83427_9989_4b67_b740_30f7f99836a4.slice/crio-4ac22426aaac5453af2b30268aae88f7ab85adb188ea880f64cf74c897507e6c WatchSource:0}: Error finding container 4ac22426aaac5453af2b30268aae88f7ab85adb188ea880f64cf74c897507e6c: Status 404 returned error can't find the container with id 4ac22426aaac5453af2b30268aae88f7ab85adb188ea880f64cf74c897507e6c Jul 15 13:11:21.262677 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:21.262632 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nr7q/must-gather-wqwxw" event={"ID":"cfe83427-9989-4b67-b740-30f7f99836a4","Type":"ContainerStarted","Data":"7bcf858193a416bdfe55b6483937979e8b44e60476d76f033303faec242da631"} Jul 15 13:11:21.263035 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:21.262688 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nr7q/must-gather-wqwxw" event={"ID":"cfe83427-9989-4b67-b740-30f7f99836a4","Type":"ContainerStarted","Data":"4ac22426aaac5453af2b30268aae88f7ab85adb188ea880f64cf74c897507e6c"} Jul 15 13:11:22.275727 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:22.275678 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nr7q/must-gather-wqwxw" event={"ID":"cfe83427-9989-4b67-b740-30f7f99836a4","Type":"ContainerStarted","Data":"64b05d4b4c2c762dd24267f0d2671f740e8443415cb1b5ddbb60f88935bf4480"} Jul 15 13:11:22.298232 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:22.298170 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9nr7q/must-gather-wqwxw" podStartSLOduration=2.463776409 podStartE2EDuration="3.298148426s" totalImagesPullingTime="834.372017ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:11:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:11:22.294633677 +0000 UTC m=+2882.845393215" watchObservedRunningTime="2026-07-15 13:11:22.298148426 +0000 UTC m=+2882.848907959" Jul 15 13:11:22.745509 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:22.745479 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5qqkj_741e3629-2b48-4694-a392-189d31991a8f/global-pull-secret-syncer/0.log" Jul 15 13:11:22.918762 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:22.918722 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m2qqp_cdc00c95-fb22-4c7d-858d-f6aac54291cc/konnectivity-agent/0.log" Jul 15 13:11:23.034620 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:23.034537 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-131.ec2.internal_19c3823af9989bc31c938283a398f768/haproxy/0.log" Jul 15 13:11:27.430685 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.430650 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76_27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f/extract/0.log" Jul 15 13:11:27.457348 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.457318 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76_27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f/util/0.log" Jul 15 13:11:27.490389 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.490357 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j4s76_27ed7e13-7fcf-4906-a5dc-0f4ea8e8ce5f/pull/0.log" Jul 15 13:11:27.524763 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.524680 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2_23c61466-535d-433d-ba92-d9eaf1684a9e/extract/0.log" Jul 15 13:11:27.550580 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.550546 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2_23c61466-535d-433d-ba92-d9eaf1684a9e/util/0.log" Jul 15 13:11:27.584861 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.584788 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05lcf2_23c61466-535d-433d-ba92-d9eaf1684a9e/pull/0.log" Jul 15 13:11:27.617176 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.617148 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6_5596d224-0d77-437a-ad3d-971e8c95b127/extract/0.log" Jul 15 13:11:27.640099 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.640067 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6_5596d224-0d77-437a-ad3d-971e8c95b127/util/0.log" Jul 15 13:11:27.668254 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.668228 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2nx6_5596d224-0d77-437a-ad3d-971e8c95b127/pull/0.log" Jul 15 13:11:27.703003 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.702967 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg_914c99bc-2302-4dc1-bf11-028d24766371/extract/0.log" Jul 15 13:11:27.733396 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.733372 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg_914c99bc-2302-4dc1-bf11-028d24766371/util/0.log" Jul 15 13:11:27.762157 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:27.762133 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g88fg_914c99bc-2302-4dc1-bf11-028d24766371/pull/0.log" Jul 15 13:11:28.172237 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:28.172008 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c67f569c6-nftv7_0f9af326-4b15-440d-9e94-d3aa668fbfa4/authorino/0.log" Jul 15 13:11:28.208639 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:28.207915 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-58dzq_9709b468-dc18-488e-8931-3f51e1318abc/manager/0.log" Jul 15 13:11:28.234683 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:28.234652 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6rbm2_8ff0870c-aa56-4af7-b340-28e9470df6ae/manager/0.log" Jul 15 13:11:28.319320 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:28.319259 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dmhq5_9ec0e4a2-d10d-4e20-adbb-75f805848c47/registry-server/0.log" Jul 15 13:11:28.504720 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:28.504540 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dpgl5_0cc9cc6f-001e-4852-8c91-233785c0d043/manager/0.log" Jul 15 13:11:30.479779 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:30.479741 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-wpxm9_a523205d-1b6d-4987-a42a-d3de5c5481de/cluster-monitoring-operator/0.log" Jul 15 13:11:30.754971 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:30.754892 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkk8f_08e0f63a-6f49-4a8e-b7e7-003b23ac63f5/node-exporter/0.log" Jul 15 13:11:30.776840 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:30.776815 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkk8f_08e0f63a-6f49-4a8e-b7e7-003b23ac63f5/kube-rbac-proxy/0.log" Jul 15 13:11:30.799730 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:30.799691 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkk8f_08e0f63a-6f49-4a8e-b7e7-003b23ac63f5/init-textfile/0.log" Jul 15 13:11:32.785697 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:32.785668 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-nnjv2_4492539b-db0b-4b2d-98fa-18d543de1458/networking-console-plugin/0.log" Jul 15 13:11:32.902357 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:32.902314 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm"] Jul 15 13:11:32.907932 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:32.907902 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:32.924593 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:32.924560 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm"] Jul 15 13:11:33.071394 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.071209 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-lib-modules\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.071394 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.071290 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7vx4\" (UniqueName: \"kubernetes.io/projected/5e69843d-7bce-4370-b059-1e52252ebfdd-kube-api-access-t7vx4\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.071634 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.071427 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-podres\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.071634 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.071476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-proc\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.071634 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.071576 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-sys\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.172374 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.172335 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vx4\" (UniqueName: \"kubernetes.io/projected/5e69843d-7bce-4370-b059-1e52252ebfdd-kube-api-access-t7vx4\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.172749 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.172726 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-podres\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.172932 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.172916 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-proc\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.173110 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.172918 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-podres\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.173269 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.172970 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-proc\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.173370 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.173118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-sys\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.173535 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.173521 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-lib-modules\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.173777 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.173757 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-lib-modules\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.173920 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.173176 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e69843d-7bce-4370-b059-1e52252ebfdd-sys\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.182864 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.182835 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7vx4\" (UniqueName: \"kubernetes.io/projected/5e69843d-7bce-4370-b059-1e52252ebfdd-kube-api-access-t7vx4\") pod \"perf-node-gather-daemonset-gsnxm\" (UID: \"5e69843d-7bce-4370-b059-1e52252ebfdd\") " pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.226073 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.226042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:33.421721 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.419734 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm"] Jul 15 13:11:33.421721 ip-10-0-142-131 kubenswrapper[2559]: W0715 13:11:33.421458 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5e69843d_7bce_4370_b059_1e52252ebfdd.slice/crio-5057775654d3a0b793b4e2014705e973c83c2da73ac03f1582060454bfa14d43 WatchSource:0}: Error finding container 5057775654d3a0b793b4e2014705e973c83c2da73ac03f1582060454bfa14d43: Status 404 returned error can't find the container with id 5057775654d3a0b793b4e2014705e973c83c2da73ac03f1582060454bfa14d43 Jul 15 13:11:33.474848 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.474821 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/1.log" Jul 15 13:11:33.482024 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.482002 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xg6rc_7be01789-354a-4605-ab90-c7813324c30d/console-operator/2.log" Jul 15 13:11:33.982057 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:33.982031 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-744ff8f984-2lmqx_5fee2079-f9fb-4d20-8c82-85b274a83537/console/0.log" Jul 15 13:11:34.025407 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:34.025380 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-djq2j_d765379d-99a6-4043-95cb-2fe71731a3ef/download-server/0.log" Jul 15 13:11:34.420093 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:34.419989 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" event={"ID":"5e69843d-7bce-4370-b059-1e52252ebfdd","Type":"ContainerStarted","Data":"b9a1128cf0750558558805d762bf452370683c99af24a610444817b78bda957c"} Jul 15 13:11:34.420093 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:34.420053 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" event={"ID":"5e69843d-7bce-4370-b059-1e52252ebfdd","Type":"ContainerStarted","Data":"5057775654d3a0b793b4e2014705e973c83c2da73ac03f1582060454bfa14d43"} Jul 15 13:11:34.420328 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:34.420194 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:34.441650 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:34.441586 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" podStartSLOduration=2.441564812 podStartE2EDuration="2.441564812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:11:34.439562811 +0000 UTC m=+2894.990322368" watchObservedRunningTime="2026-07-15 13:11:34.441564812 +0000 UTC m=+2894.992324344" Jul 15 13:11:34.665195 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:34.665153 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-p5s6c_97c63a62-8a37-4495-90c9-0944d7fdb759/volume-data-source-validator/0.log" Jul 15 13:11:35.779983 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:35.779944 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jm9h6_62c2d3bc-8cac-427f-a8da-bec575ab8b3e/dns/0.log" Jul 15 13:11:35.802928 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:35.802897 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jm9h6_62c2d3bc-8cac-427f-a8da-bec575ab8b3e/kube-rbac-proxy/0.log" Jul 15 13:11:36.015568 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:36.015533 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nr9tq_489c2f9f-1652-4042-9366-dbb76fac0840/dns-node-resolver/0.log" Jul 15 13:11:36.687019 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:36.686987 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xksn9_90572c3f-6cb5-4c41-8b12-222f3041b25d/node-ca/0.log" Jul 15 13:11:37.657954 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.657921 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fqz8m_40c38105-86fe-4c9b-9c8c-8958a54f7203/istio-proxy/0.log" Jul 15 13:11:37.788344 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.788317 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9797f867b-j97px_ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3/kube-auth-proxy/0.log" Jul 15 13:11:37.844527 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.844494 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/0.log" Jul 15 13:11:37.881338 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.881308 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/1.log" Jul 15 13:11:37.916656 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.916579 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-xnldf_1b0dabbe-e728-4305-89c1-6425a32e3918/payload-pre-processing/0.log" Jul 15 13:11:37.947914 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.947884 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fmfd5_783c2fa1-4ace-4f75-b866-f2f37de9fee9/payload-processing/0.log" Jul 15 13:11:37.970064 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:37.970036 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c7ddc6b-857qm_1effb694-29c1-4b54-8822-604b929a4284/router/0.log" Jul 15 13:11:38.662348 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:38.662321 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8tw9h_f35b6a0c-30d5-486c-a98b-adb08a528ea7/serve-healthcheck-canary/0.log" Jul 15 13:11:39.225499 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:39.225468 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-bmp27_1e360f0c-fc2b-4915-92ab-61a3c4b63e71/insights-operator/0.log" Jul 15 13:11:39.226334 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:39.226312 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-bmp27_1e360f0c-fc2b-4915-92ab-61a3c4b63e71/insights-operator/1.log" Jul 15 13:11:39.469880 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:39.469847 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vfr2c_60110441-30ee-4331-9881-17af64ac1a99/kube-rbac-proxy/0.log" Jul 15 13:11:39.496178 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:39.496101 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vfr2c_60110441-30ee-4331-9881-17af64ac1a99/exporter/0.log" Jul 15 13:11:39.521179 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:39.521134 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vfr2c_60110441-30ee-4331-9881-17af64ac1a99/extractor/0.log" Jul 15 13:11:40.438942 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:40.438913 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9nr7q/perf-node-gather-daemonset-gsnxm" Jul 15 13:11:41.755329 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:41.755304 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tkrr4_d8858c84-c69a-4b01-bdbc-38b065a7508b/manager/0.log" Jul 15 13:11:41.933003 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:41.932972 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xm9ck_f9d8e1c7-3a6e-4dc6-ba71-eea13b56b466/manager/0.log" Jul 15 13:11:42.081847 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:42.081792 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fc57dd599-d9rk9_87e69da8-f9ac-45fd-befa-4a707a06ff6b/manager/0.log" Jul 15 13:11:42.104944 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:42.104850 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vff75_e6cdba19-5aaf-4219-a329-1e733a2391db/server/0.log" Jul 15 13:11:42.202201 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:42.201112 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cwkbf_ecb74447-252c-4ce1-978b-c1cba7ddc4fa/manager/0.log" Jul 15 13:11:42.231777 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:42.231745 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-654f68fd79-6zqrk_00e13959-81d2-47d0-962f-d4db59be27da/manager/0.log" Jul 15 13:11:42.255663 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:42.255612 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-654f68fd79-6zqrk_00e13959-81d2-47d0-962f-d4db59be27da/copy-manifests/0.log" Jul 15 13:11:43.450669 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.450565 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fqz8m_40c38105-86fe-4c9b-9c8c-8958a54f7203/istio-proxy/0.log" Jul 15 13:11:43.575714 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.575685 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9797f867b-j97px_ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3/kube-auth-proxy/0.log" Jul 15 13:11:43.616994 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.616964 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/0.log" Jul 15 13:11:43.655987 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.655955 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/1.log" Jul 15 13:11:43.678589 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.678557 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-xnldf_1b0dabbe-e728-4305-89c1-6425a32e3918/payload-pre-processing/0.log" Jul 15 13:11:43.698247 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.698213 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fmfd5_783c2fa1-4ace-4f75-b866-f2f37de9fee9/payload-processing/0.log" Jul 15 13:11:43.718928 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.718895 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c7ddc6b-857qm_1effb694-29c1-4b54-8822-604b929a4284/router/0.log" Jul 15 13:11:43.825379 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:43.825322 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zfbvx_27155296-5ccf-4287-9c78-87f59c7585c6/openshift-lws-operator/0.log" Jul 15 13:11:44.940999 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:44.940971 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fqz8m_40c38105-86fe-4c9b-9c8c-8958a54f7203/istio-proxy/0.log" Jul 15 13:11:45.065207 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:45.065181 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9797f867b-j97px_ec7b7ea2-276d-4572-b1b2-2cb4629b5ac3/kube-auth-proxy/0.log" Jul 15 13:11:45.119070 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:45.118988 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/0.log" Jul 15 13:11:45.162702 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:45.162668 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j6q6z_2bc79424-ac1c-4dd0-9284-610be84542ec/istio-proxy/1.log" Jul 15 13:11:45.186996 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:45.186959 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-xnldf_1b0dabbe-e728-4305-89c1-6425a32e3918/payload-pre-processing/0.log" Jul 15 13:11:45.208887 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:45.208855 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fmfd5_783c2fa1-4ace-4f75-b866-f2f37de9fee9/payload-processing/0.log" Jul 15 13:11:45.222990 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:45.222967 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c7ddc6b-857qm_1effb694-29c1-4b54-8822-604b929a4284/router/0.log" Jul 15 13:11:49.216478 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:49.216447 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-84h6r_daf7b2a0-74e0-4a82-9131-0f262df3912b/migrator/0.log" Jul 15 13:11:49.237668 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:49.237639 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-84h6r_daf7b2a0-74e0-4a82-9131-0f262df3912b/graceful-termination/0.log" Jul 15 13:11:49.611835 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:49.611744 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-dp9wg_19bf4b2b-2ac1-4b62-a476-df4f617e74f8/kube-storage-version-migrator-operator/1.log" Jul 15 13:11:49.612650 ip-10-0-142-131 kubenswrapper[2559]: I0715 13:11:49.612632 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-dp9wg_19bf4b2b-2ac1-4b62-a476-df4f617e74f8/kube-storage-version-migrator-operator/0.log"