Jun 09 00:07:28.531937 ip-10-0-143-86 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 09 00:07:28.531949 ip-10-0-143-86 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 09 00:07:28.531957 ip-10-0-143-86 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 09 00:07:28.532159 ip-10-0-143-86 systemd[1]: Failed to start Kubernetes Kubelet. Jun 09 00:07:38.702831 ip-10-0-143-86 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 09 00:07:38.702851 ip-10-0-143-86 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0436e577dade4ebdbefad45f3d0395ab -- Jun 09 00:10:36.658443 ip-10-0-143-86 systemd[1]: Starting Kubernetes Kubelet... Jun 09 00:10:37.058099 ip-10-0-143-86 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 00:10:37.058099 ip-10-0-143-86 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 00:10:37.058099 ip-10-0-143-86 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 00:10:37.058099 ip-10-0-143-86 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 00:10:37.058099 ip-10-0-143-86 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 00:10:37.059181 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.059091 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 00:10:37.063484 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063465 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063485 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063489 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063493 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063498 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063501 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063504 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063507 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063510 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063513 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063515 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063518 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063520 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063523 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063526 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063528 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063531 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063533 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 00:10:37.063535 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063536 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063547 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063550 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063553 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063556 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063558 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063561 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063563 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063566 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063569 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063571 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063574 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063577 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063579 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063582 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063585 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063587 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063589 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063592 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063594 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 00:10:37.063996 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063600 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063603 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063605 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063608 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063610 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063613 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063615 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063617 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063620 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063623 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063625 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063627 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063630 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063634 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063640 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063643 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063645 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063648 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063651 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063653 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 00:10:37.064550 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063655 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063658 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063660 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063663 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063665 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063668 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063670 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063674 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063677 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063680 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063684 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063687 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063689 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063692 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063695 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063697 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063700 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063702 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063705 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 00:10:37.065063 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063707 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063710 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063713 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063715 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063718 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063720 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063723 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063728 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063731 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063734 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063737 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063740 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063742 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063745 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063747 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063750 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063752 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063755 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063758 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063760 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 00:10:37.065664 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063763 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.063765 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064517 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064524 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064528 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064530 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064533 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064537 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064540 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064542 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064545 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064547 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064550 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064553 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064556 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064558 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064561 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064564 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064566 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064571 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 00:10:37.066204 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064574 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064577 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064579 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064582 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064585 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064587 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064590 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064592 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064613 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064617 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064621 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064624 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064627 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064630 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064633 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064635 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064638 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064641 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064643 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064645 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 00:10:37.066740 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064649 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064651 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064654 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064656 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064659 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064661 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064664 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064666 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064669 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064672 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064676 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064681 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064684 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064687 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064690 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064694 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064698 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064702 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064704 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 00:10:37.067274 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064707 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064709 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064712 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064714 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064717 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064720 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064722 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064725 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064727 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064730 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064732 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064734 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064737 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064739 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064745 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064749 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064752 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064755 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064758 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 00:10:37.067764 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064760 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064763 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064765 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064768 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064770 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064775 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064777 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064780 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064783 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064785 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064788 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064790 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064792 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064795 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064798 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064800 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064803 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064805 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064808 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064810 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 00:10:37.068293 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064813 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064815 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.064818 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065256 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065261 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065263 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065266 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065270 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065273 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065275 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065278 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065281 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065285 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065289 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065292 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065296 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065299 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065303 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065306 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 00:10:37.068808 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065309 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065311 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065314 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065316 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065319 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065321 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065324 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065326 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065328 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065331 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065333 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065336 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065338 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065341 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065343 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065346 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065349 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065351 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065354 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065357 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 00:10:37.069326 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065361 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065365 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065367 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065370 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065373 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065375 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065378 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065381 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065383 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065385 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065389 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065392 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065394 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065397 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065399 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065401 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065404 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065406 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065409 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065411 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 00:10:37.069855 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065414 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065416 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065419 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065422 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065424 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065426 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065429 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065432 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065434 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065437 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065439 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065441 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065444 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065446 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065449 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065453 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065457 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065459 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065462 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065465 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 00:10:37.070438 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065467 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065470 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065476 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065479 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065482 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065484 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065487 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065490 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065493 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065495 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065498 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065501 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065503 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065506 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065508 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065511 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065513 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065516 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065518 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065521 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 00:10:37.070964 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065523 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065526 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.065528 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065604 2550 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065612 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065627 2550 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065633 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065641 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065645 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065649 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065654 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065657 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065660 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065664 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065668 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065672 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065674 2550 flags.go:64] FLAG: --cgroup-root="" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065677 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065681 2550 flags.go:64] FLAG: --client-ca-file="" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065684 2550 flags.go:64] FLAG: --cloud-provider="external" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065687 2550 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065692 2550 flags.go:64] FLAG: --cluster-domain="" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065695 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 00:10:37.071501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065698 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065702 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065706 2550 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065709 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065712 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065715 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065718 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065721 2550 flags.go:64] FLAG: --contention-profiling="false" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065724 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065727 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065730 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065733 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065736 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065739 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065742 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065745 2550 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065748 2550 flags.go:64] FLAG: --enable-server="true" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065751 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065755 2550 flags.go:64] FLAG: --event-burst="100" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065758 2550 flags.go:64] FLAG: --event-qps="50" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065761 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065763 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065766 2550 flags.go:64] FLAG: --eviction-hard="" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065771 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 00:10:37.072101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065774 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065777 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065780 2550 flags.go:64] FLAG: --eviction-soft="" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065782 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065785 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065788 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065791 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065794 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065796 2550 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065801 2550 flags.go:64] FLAG: --feature-gates="" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065805 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065808 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065811 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065813 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065816 2550 flags.go:64] FLAG: --healthz-port="10248" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065819 2550 flags.go:64] FLAG: --help="false" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065822 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065825 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065828 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065830 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065834 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065837 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065839 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 00:10:37.072756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065842 2550 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065845 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065848 2550 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065851 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065854 2550 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065857 2550 flags.go:64] FLAG: --kube-reserved="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065860 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065862 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065866 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065869 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065872 2550 flags.go:64] FLAG: --lock-file="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065874 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065877 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065880 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065885 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065888 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065891 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065893 2550 flags.go:64] FLAG: --logging-format="text" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065897 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065901 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065903 2550 flags.go:64] FLAG: --manifest-url="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065906 2550 flags.go:64] FLAG: --manifest-url-header="" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065910 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065913 2550 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065917 2550 flags.go:64] FLAG: --max-pods="110" Jun 09 00:10:37.073424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065920 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065923 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065926 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065929 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065932 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065934 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065937 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065944 2550 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065946 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065949 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065952 2550 flags.go:64] FLAG: --pod-cidr="" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065956 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065961 2550 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065964 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065967 2550 flags.go:64] FLAG: --pods-per-core="0" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065970 2550 flags.go:64] FLAG: --port="10250" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065974 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065977 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c93b0f7b7005c89f" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065980 2550 flags.go:64] FLAG: --qos-reserved="" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065983 2550 flags.go:64] FLAG: --read-only-port="10255" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065986 2550 flags.go:64] FLAG: --register-node="true" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065989 2550 flags.go:64] FLAG: --register-with-taints="" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065993 2550 flags.go:64] FLAG: --registry-burst="10" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065996 2550 flags.go:64] FLAG: --registry-qps="5" Jun 09 00:10:37.074148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.065999 2550 flags.go:64] FLAG: --reserved-cpus="" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066003 2550 flags.go:64] FLAG: --reserved-memory="" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066006 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066009 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066013 2550 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066015 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066018 2550 flags.go:64] FLAG: --runonce="false" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066021 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066024 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066027 2550 flags.go:64] FLAG: --seccomp-default="false" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066030 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066033 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066036 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066038 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066041 2550 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066044 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066047 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066050 2550 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066052 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066055 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066059 2550 flags.go:64] FLAG: --system-cgroups="" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066062 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066067 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066069 2550 flags.go:64] FLAG: --tls-cert-file="" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066090 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 00:10:37.074770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066095 2550 flags.go:64] FLAG: --tls-min-version="" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066098 2550 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066101 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066103 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066106 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066109 2550 flags.go:64] FLAG: --v="2" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066113 2550 flags.go:64] FLAG: --version="false" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066117 2550 flags.go:64] FLAG: --vmodule="" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066123 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.066126 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066252 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066255 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066258 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066261 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066264 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066266 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066269 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066273 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066277 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066280 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066283 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066304 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 00:10:37.075471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066323 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066328 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066331 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066335 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066338 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066341 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066345 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066350 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066354 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066358 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066361 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066366 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066369 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066372 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066375 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066378 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066381 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066384 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066386 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066389 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 00:10:37.076042 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066392 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066395 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066397 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066400 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066403 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066405 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066408 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066410 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066413 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066415 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066417 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066420 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066426 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066428 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066432 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066434 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066437 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066440 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066442 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066445 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 00:10:37.076598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066447 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066450 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066453 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066455 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066457 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066460 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066465 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066469 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066472 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066475 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066478 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066480 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066483 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066485 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066488 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066491 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066494 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066496 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066499 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 00:10:37.077151 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066501 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066504 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066507 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066510 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066512 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066515 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066519 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066522 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066524 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066527 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066530 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066532 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066535 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066537 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066540 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066543 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066545 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066548 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066550 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066553 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 00:10:37.077653 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066555 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066557 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066560 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066562 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066565 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066567 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066570 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.066573 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.067298 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.074189 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.074206 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.074915 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.074922 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.074981 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.074986 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 00:10:37.078198 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.074990 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.074993 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.074996 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.074999 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075002 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075004 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075007 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075009 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075012 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075014 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075017 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075021 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075025 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075027 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075030 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075032 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075035 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075038 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075041 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 00:10:37.078598 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075044 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075046 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075049 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075052 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075054 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075057 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075059 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075062 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075065 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075068 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075070 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075073 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075093 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075097 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075100 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075103 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075106 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075109 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075112 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075114 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 00:10:37.079112 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075117 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075120 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075122 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075124 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075127 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075129 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075132 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075134 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075137 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075139 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075142 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075144 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075146 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075149 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075152 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075154 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075157 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075159 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075162 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075164 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 00:10:37.079644 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075168 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075171 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075174 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075177 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075179 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075182 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075184 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075187 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075190 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075193 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075195 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075198 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075200 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075202 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075205 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075207 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075210 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075212 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075214 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075217 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 00:10:37.080178 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075219 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075223 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075227 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075230 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075234 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075237 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075240 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075243 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075246 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075249 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075251 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075254 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075257 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075260 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075262 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075265 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075268 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 00:10:37.080707 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075270 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.075276 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075402 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075407 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075411 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075413 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075417 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075419 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075422 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075424 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075427 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075429 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075432 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075435 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 00:10:37.081172 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075439 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075442 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075445 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075448 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075450 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075453 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075456 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075459 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075462 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075466 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075469 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075471 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075474 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075477 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075479 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075482 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075485 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075488 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075490 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075493 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 00:10:37.081580 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075495 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075498 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075500 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075504 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075506 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075508 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075511 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075514 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075516 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075519 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075522 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075525 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075528 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075530 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075532 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075535 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075537 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075539 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075542 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075545 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 00:10:37.082126 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075547 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075550 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075552 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075555 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075557 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075560 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075562 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075565 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075567 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075570 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075572 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075575 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075577 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075579 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075582 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075584 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075587 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075589 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075592 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075594 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 00:10:37.082837 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075596 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075599 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075601 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075604 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075606 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075609 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075611 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075613 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075616 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075618 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075621 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075624 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075626 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075629 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075631 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075633 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075636 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075639 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075641 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 00:10:37.083675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075644 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075646 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075649 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075651 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075653 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075656 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075659 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:37.075661 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.075666 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.075778 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.077773 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.079216 2550 server.go:1019] "Starting client certificate rotation" Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.079316 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 00:10:37.084341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.079362 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 00:10:37.084687 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.084358 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 00:10:37.088871 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.087390 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 00:10:37.098829 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.098810 2550 log.go:25] "Validated CRI v1 runtime API" Jun 09 00:10:37.101603 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.101588 2550 log.go:25] "Validated CRI v1 image API" Jun 09 00:10:37.102542 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.102527 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 00:10:37.106530 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.106511 2550 fs.go:135] Filesystem UUIDs: map[2e2b4682-189d-4f0a-a1c0-4bc65eb8a522:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e57daf1c-2ac5-4c4b-8cdc-e725f859296d:/dev/nvme0n1p3] Jun 09 00:10:37.106580 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.106534 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 00:10:37.111652 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.111537 2550 manager.go:216] Machine: {Timestamp:2026-06-09 00:10:37.110209283 +0000 UTC m=+0.334606540 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105063 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26196a2da521bc7d86513b55ed1c96 SystemUUID:ec26196a-2da5-21bc-7d86-513b55ed1c96 BootID:0436e577-dade-4ebd-befa-d45f3d0395ab Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:a9:b6:7d:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:a9:b6:7d:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:70:aa:06:18:8b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 00:10:37.111652 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.111650 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 00:10:37.111771 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.111698 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 00:10:37.111771 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.111740 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 00:10:37.114214 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.114185 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 00:10:37.114360 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.114218 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-86.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 00:10:37.115463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.115451 2550 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 00:10:37.115508 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.115467 2550 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 00:10:37.115508 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.115475 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 00:10:37.116177 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.116166 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 00:10:37.116990 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.116979 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 00:10:37.119915 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.119902 2550 state_mem.go:36] "Initialized new in-memory state store" Jun 09 00:10:37.120042 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120032 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 00:10:37.120091 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120061 2550 kubelet.go:486] "Attempting to sync node with API server" Jun 09 00:10:37.120136 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120093 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 00:10:37.120136 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120109 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 00:10:37.120136 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120121 2550 kubelet.go:398] "Adding apiserver pod source" Jun 09 00:10:37.120136 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120130 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 00:10:37.120518 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120506 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 00:10:37.120557 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.120529 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 00:10:37.122764 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.122745 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 00:10:37.127521 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.127501 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 00:10:37.127626 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.127535 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 00:10:37.128546 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128534 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128552 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128559 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128565 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128573 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128578 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128587 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 00:10:37.128592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128592 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 00:10:37.128788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128598 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 00:10:37.128788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128604 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 00:10:37.128788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128612 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 00:10:37.128788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.128617 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 00:10:37.129955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.129945 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 00:10:37.129955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.129956 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 00:10:37.133849 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.133834 2550 server.go:1268] "Started kubelet" Jun 09 00:10:37.134009 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.133978 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 00:10:37.134143 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.134096 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 00:10:37.134219 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.134168 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 00:10:37.134762 ip-10-0-143-86 systemd[1]: Started Kubernetes Kubelet. Jun 09 00:10:37.135881 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.135847 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-86.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 00:10:37.136164 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.136149 2550 server.go:310] "Adding debug handlers to kubelet server" Jun 09 00:10:37.137101 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.137059 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 00:10:37.137101 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.137062 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 00:10:37.139222 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.139204 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 00:10:37.139812 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.138949 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-86.ec2.internal.18b7412c862d4474 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-86.ec2.internal,UID:ip-10-0-143-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-86.ec2.internal,},FirstTimestamp:2026-06-09 00:10:37.133800564 +0000 UTC m=+0.358197822,LastTimestamp:2026-06-09 00:10:37.133800564 +0000 UTC m=+0.358197822,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-86.ec2.internal,}" Jun 09 00:10:37.144942 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.144903 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 00:10:37.145291 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.145275 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 00:10:37.145390 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.145300 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 00:10:37.145585 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.145396 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.146003 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.145983 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 00:10:37.146093 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146008 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 00:10:37.146182 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146146 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 00:10:37.146284 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146239 2550 reconstruct.go:97] "Volume reconstruction finished" Jun 09 00:10:37.146284 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146249 2550 reconciler.go:29] "Reconciler: start to sync state" Jun 09 00:10:37.146432 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146416 2550 factory.go:55] Registering systemd factory Jun 09 00:10:37.146487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146435 2550 factory.go:223] Registration of the systemd container factory successfully Jun 09 00:10:37.146681 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146661 2550 factory.go:153] Registering CRI-O factory Jun 09 00:10:37.146681 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146681 2550 factory.go:223] Registration of the crio container factory successfully Jun 09 00:10:37.146815 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146735 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 00:10:37.146815 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146764 2550 factory.go:103] Registering Raw factory Jun 09 00:10:37.146815 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.146780 2550 manager.go:1196] Started watching for new ooms in manager Jun 09 00:10:37.147304 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.147288 2550 manager.go:349] Starting recovery of all containers Jun 09 00:10:37.155732 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.155694 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 09 00:10:37.156840 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.156796 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 09 00:10:37.158575 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.158557 2550 manager.go:354] Recovery completed Jun 09 00:10:37.164002 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.163982 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.166217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166190 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.166298 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166232 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.166298 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166243 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.166806 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166790 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 00:10:37.166880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166807 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 00:10:37.166880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166826 2550 state_mem.go:36] "Initialized new in-memory state store" Jun 09 00:10:37.166942 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.166825 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.167651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.167633 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.167700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.167663 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.167700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.167673 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.167924 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.167829 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-86.ec2.internal.18b7412c881bebba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-86.ec2.internal,UID:ip-10-0-143-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-86.ec2.internal,},FirstTimestamp:2026-06-09 00:10:37.16621817 +0000 UTC m=+0.390615429,LastTimestamp:2026-06-09 00:10:37.16621817 +0000 UTC m=+0.390615429,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-86.ec2.internal,}" Jun 09 00:10:37.168339 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.168325 2550 policy_none.go:49] "None policy: Start" Jun 09 00:10:37.168399 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.168342 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 00:10:37.168399 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.168353 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 00:10:37.169594 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.169582 2550 policy_none.go:47] "Start" Jun 09 00:10:37.175456 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.175432 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kmvkw" Jun 09 00:10:37.176520 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.176449 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-86.ec2.internal.18b7412c881c3a81 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-86.ec2.internal,UID:ip-10-0-143-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-86.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-86.ec2.internal,},FirstTimestamp:2026-06-09 00:10:37.166238337 +0000 UTC m=+0.390635595,LastTimestamp:2026-06-09 00:10:37.166238337 +0000 UTC m=+0.390635595,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-86.ec2.internal,}" Jun 09 00:10:37.179293 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.179226 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-86.ec2.internal.18b7412c881c5e21 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-86.ec2.internal,UID:ip-10-0-143-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-86.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-86.ec2.internal,},FirstTimestamp:2026-06-09 00:10:37.166247457 +0000 UTC m=+0.390644715,LastTimestamp:2026-06-09 00:10:37.166247457 +0000 UTC m=+0.390644715,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-86.ec2.internal,}" Jun 09 00:10:37.180650 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.180634 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kmvkw" Jun 09 00:10:37.183983 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.183908 2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-86.ec2.internal.18b7412c881bebba\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-86.ec2.internal.18b7412c881bebba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-86.ec2.internal,UID:ip-10-0-143-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-86.ec2.internal,},FirstTimestamp:2026-06-09 00:10:37.16621817 +0000 UTC m=+0.390615429,LastTimestamp:2026-06-09 00:10:37.167649875 +0000 UTC m=+0.392047137,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-86.ec2.internal,}" Jun 09 00:10:37.205095 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205056 2550 manager.go:337] "Starting Device Plugin manager" Jun 09 00:10:37.205212 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.205118 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 00:10:37.205212 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205132 2550 server.go:87] "Starting device plugin registration server" Jun 09 00:10:37.205418 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205406 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 00:10:37.205456 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205421 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 00:10:37.205580 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205540 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 00:10:37.205675 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205652 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 00:10:37.205675 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.205671 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 00:10:37.206207 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.206175 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 00:10:37.206308 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.206234 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.227763 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.227725 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 00:10:37.229993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.229038 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 00:10:37.229993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.229115 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 00:10:37.229993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.229147 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 00:10:37.229993 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.229245 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 00:10:37.231060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.231043 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 00:10:37.306497 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.306452 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.309064 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.309011 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.309064 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.309044 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.309064 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.309055 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.309211 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.309099 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.317768 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.317749 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.317897 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.317775 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-86.ec2.internal\": node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.329500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.329476 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal"] Jun 09 00:10:37.329585 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.329543 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.330575 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.330560 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.330650 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.330596 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.330650 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.330609 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.331876 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.331863 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.331991 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.331974 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.332051 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332025 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.332578 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332561 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.332663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332583 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.332663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332592 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.332663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332615 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.332663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332638 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.332663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.332653 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.333954 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.333939 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.334027 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.333965 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:37.334693 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.334678 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:37.334766 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.334710 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:37.334766 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.334724 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:37.344384 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.344367 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.347551 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.347533 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ea97a5bdfd910401bb9da830322516f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal\" (UID: \"8ea97a5bdfd910401bb9da830322516f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.347638 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.347564 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2ca2fd9e0ea5addc9b16951a720ff81-config\") pod \"kube-apiserver-proxy-ip-10-0-143-86.ec2.internal\" (UID: \"b2ca2fd9e0ea5addc9b16951a720ff81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.347638 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.347588 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ea97a5bdfd910401bb9da830322516f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal\" (UID: \"8ea97a5bdfd910401bb9da830322516f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.366146 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.366123 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-86.ec2.internal\" not found" node="ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.371071 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.371053 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-86.ec2.internal\" not found" node="ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.444923 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.444884 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.448178 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.448157 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2ca2fd9e0ea5addc9b16951a720ff81-config\") pod \"kube-apiserver-proxy-ip-10-0-143-86.ec2.internal\" (UID: \"b2ca2fd9e0ea5addc9b16951a720ff81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.448288 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.448192 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ea97a5bdfd910401bb9da830322516f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal\" (UID: \"8ea97a5bdfd910401bb9da830322516f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.448288 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.448222 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ea97a5bdfd910401bb9da830322516f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal\" (UID: \"8ea97a5bdfd910401bb9da830322516f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.448288 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.448258 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2ca2fd9e0ea5addc9b16951a720ff81-config\") pod \"kube-apiserver-proxy-ip-10-0-143-86.ec2.internal\" (UID: \"b2ca2fd9e0ea5addc9b16951a720ff81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.448288 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.448264 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ea97a5bdfd910401bb9da830322516f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal\" (UID: \"8ea97a5bdfd910401bb9da830322516f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.448432 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.448257 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ea97a5bdfd910401bb9da830322516f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal\" (UID: \"8ea97a5bdfd910401bb9da830322516f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.545367 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.545335 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.646156 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.646092 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.670529 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.670502 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.674978 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:37.674953 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:37.746705 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.746659 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.847094 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.847049 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:37.947626 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:37.947521 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:38.047989 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:38.047966 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:38.078313 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.078279 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 00:10:38.079227 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.078460 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 00:10:38.139692 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:38.139656 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea97a5bdfd910401bb9da830322516f.slice/crio-c17f97f0d6a6f2bda64aca7f511b440620bb1433925d704ed2113c0f657a6a95 WatchSource:0}: Error finding container c17f97f0d6a6f2bda64aca7f511b440620bb1433925d704ed2113c0f657a6a95: Status 404 returned error can't find the container with id c17f97f0d6a6f2bda64aca7f511b440620bb1433925d704ed2113c0f657a6a95 Jun 09 00:10:38.141263 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:38.140688 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ca2fd9e0ea5addc9b16951a720ff81.slice/crio-1870d7da4b5e3d3cc24cfb73d13a3bd5a6f1a23bc3af4476fcb8fdad7f12b877 WatchSource:0}: Error finding container 1870d7da4b5e3d3cc24cfb73d13a3bd5a6f1a23bc3af4476fcb8fdad7f12b877: Status 404 returned error can't find the container with id 1870d7da4b5e3d3cc24cfb73d13a3bd5a6f1a23bc3af4476fcb8fdad7f12b877 Jun 09 00:10:38.146509 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.146492 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 00:10:38.148091 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:38.148051 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:38.183388 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.183330 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 00:05:37 +0000 UTC" deadline="2028-03-11 16:34:38.025974232 +0000 UTC" Jun 09 00:10:38.183388 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.183380 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15400h23m59.842599407s" Jun 09 00:10:38.208668 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.208609 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 00:10:38.209580 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.209564 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientMemory" Jun 09 00:10:38.209643 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.209595 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 00:10:38.209643 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.209606 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeHasSufficientPID" Jun 09 00:10:38.233090 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.233041 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" event={"ID":"b2ca2fd9e0ea5addc9b16951a720ff81","Type":"ContainerStarted","Data":"1870d7da4b5e3d3cc24cfb73d13a3bd5a6f1a23bc3af4476fcb8fdad7f12b877"} Jun 09 00:10:38.234040 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.234018 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" event={"ID":"8ea97a5bdfd910401bb9da830322516f","Type":"ContainerStarted","Data":"c17f97f0d6a6f2bda64aca7f511b440620bb1433925d704ed2113c0f657a6a95"} Jun 09 00:10:38.248415 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:38.248384 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:38.348876 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:38.348841 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:38.358489 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.358466 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 00:10:38.449634 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:38.449596 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-86.ec2.internal\" not found" Jun 09 00:10:38.498932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.498852 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 00:10:38.535240 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.535212 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 00:10:38.546310 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.546275 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" Jun 09 00:10:38.556585 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.556552 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 00:10:38.557574 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.557521 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" Jun 09 00:10:38.567059 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:38.567008 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 00:10:39.120907 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.120872 2550 apiserver.go:52] "Watching apiserver" Jun 09 00:10:39.126860 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.126832 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 00:10:39.127251 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.127230 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-g9lwf","openshift-network-diagnostics/network-check-target-jbvbz","kube-system/konnectivity-agent-jzrmb","kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal","openshift-image-registry/node-ca-nj5z7","openshift-multus/network-metrics-daemon-vx6qc","openshift-network-operator/iptables-alerter-4dqc4","openshift-ovn-kubernetes/ovnkube-node-wks97","kube-system/global-pull-secret-syncer-x8h82","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7","openshift-cluster-node-tuning-operator/tuned-h6krf","openshift-dns/node-resolver-g97wc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal","openshift-multus/multus-additional-cni-plugins-rt7rv"] Jun 09 00:10:39.128773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.128753 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.130240 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.130222 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.131093 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.131056 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 00:10:39.131188 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.131060 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.131188 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.131067 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.131293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.131069 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zrk4f\"" Jun 09 00:10:39.132114 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.132070 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-66wqx\"" Jun 09 00:10:39.132218 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.132118 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.132218 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.132142 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.132218 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.132177 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.133574 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.133556 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.134777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.134757 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 00:10:39.134777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.134774 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-whrxw\"" Jun 09 00:10:39.134924 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.134792 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 00:10:39.135116 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.135098 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.135710 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.135692 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dbznq\"" Jun 09 00:10:39.136199 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.135962 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 00:10:39.136199 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.135962 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.136199 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.135983 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.136199 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.135980 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 00:10:39.136420 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.136303 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 00:10:39.136573 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.136558 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.136645 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.136623 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:39.137005 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.136990 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.137094 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.137012 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 00:10:39.137315 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.137299 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-trr7r\"" Jun 09 00:10:39.137363 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.137330 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.137898 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.137739 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.137898 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.137801 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:39.139113 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.139093 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.140500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.140480 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:39.140592 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.140558 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:39.141501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.141476 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.141635 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.141615 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 00:10:39.141680 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.141633 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 00:10:39.142174 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.142157 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-28x2w\"" Jun 09 00:10:39.142487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.142471 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 00:10:39.142720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.142705 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 00:10:39.143067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.143008 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.144363 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.143471 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.145412 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.145395 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.145603 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.145582 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 00:10:39.145706 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.145613 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6l9zb\"" Jun 09 00:10:39.145766 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.145545 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 00:10:39.146874 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.146852 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.147913 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.147897 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.148029 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.147946 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jkcvs\"" Jun 09 00:10:39.148106 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.148035 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.148279 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.148260 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 00:10:39.148875 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.148853 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 00:10:39.148982 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.148889 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 00:10:39.148982 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.148855 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6sf7x\"" Jun 09 00:10:39.156441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156412 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.156565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156454 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-kubelet\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.156565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156481 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-hostroot\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.156565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.156565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156520 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7cp\" (UniqueName: \"kubernetes.io/projected/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-kube-api-access-fz7cp\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.156565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156538 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-serviceca\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.156565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156562 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f62773d3-957a-4951-8974-a26ee5ebd3cb-kubelet-config\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.156816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156588 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-systemd-units\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.156816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156652 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-var-lib-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.156816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156686 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-cni-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.156816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156712 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-conf-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.156816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156765 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-sys-fs\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.156816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156803 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp6r8\" (UniqueName: \"kubernetes.io/projected/ec8d62a2-d8cc-484c-a30c-be92c9776a04-kube-api-access-sp6r8\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156831 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-modprobe-d\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156863 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv6wm\" (UniqueName: \"kubernetes.io/projected/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-kube-api-access-jv6wm\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156884 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysconfig\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156906 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-kubernetes\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156929 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/546188fe-9f2f-4ea9-8775-39af993e771d-etc-tuned\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156951 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-system-cni-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156976 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-os-release\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.157018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.156999 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-system-cni-dir\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157022 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-host\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157049 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157071 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-slash\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157111 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-run-ovn-kubernetes\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157134 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-cni-bin\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157158 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157210 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b286762b-0f99-494b-9e7c-d89fbd551c00-cni-binary-copy\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157259 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-netns\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157299 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-etc-selinux\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157323 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfccj\" (UniqueName: \"kubernetes.io/projected/dd74db37-2abc-4b24-8362-f410ec8c144a-kube-api-access-bfccj\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.157352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157346 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157369 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f62773d3-957a-4951-8974-a26ee5ebd3cb-dbus\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157394 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ec8d62a2-d8cc-484c-a30c-be92c9776a04-iptables-alerter-script\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-lib-modules\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157439 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-var-lib-kubelet\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157462 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b633971-bf33-4527-b021-b8f97a6b1f69-agent-certs\") pod \"konnectivity-agent-jzrmb\" (UID: \"4b633971-bf33-4527-b021-b8f97a6b1f69\") " pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-cnibin\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157522 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-os-release\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157544 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157569 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/546188fe-9f2f-4ea9-8775-39af993e771d-tmp\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157591 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-cni-netd\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157614 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-cni-bin\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157637 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcv96\" (UniqueName: \"kubernetes.io/projected/b286762b-0f99-494b-9e7c-d89fbd551c00-kube-api-access-tcv96\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157659 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-socket-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157681 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-device-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157703 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-host\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157744 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-etc-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.157770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157777 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-env-overrides\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157803 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157838 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5wx\" (UniqueName: \"kubernetes.io/projected/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-kube-api-access-hx5wx\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157864 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk57k\" (UniqueName: \"kubernetes.io/projected/546188fe-9f2f-4ea9-8775-39af993e771d-kube-api-access-zk57k\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-kubelet\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157913 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd74db37-2abc-4b24-8362-f410ec8c144a-ovn-node-metrics-cert\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157938 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cnibin\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157961 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-systemd\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.157984 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b633971-bf33-4527-b021-b8f97a6b1f69-konnectivity-ca\") pod \"konnectivity-agent-jzrmb\" (UID: \"4b633971-bf33-4527-b021-b8f97a6b1f69\") " pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158007 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-socket-dir-parent\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158030 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-k8s-cni-cncf-io\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158054 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qjrc\" (UniqueName: \"kubernetes.io/projected/e7ba130d-8a9d-4fc7-bc44-009b00be3655-kube-api-access-8qjrc\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158100 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysctl-conf\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158122 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-sys\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158150 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52g96\" (UniqueName: \"kubernetes.io/projected/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-kube-api-access-52g96\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158174 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-run-netns\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.158510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158209 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158238 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-node-log\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158262 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-log-socket\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158284 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-ovnkube-config\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158309 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-multus-certs\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158330 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-hosts-file\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158352 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-cni-multus\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158373 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec8d62a2-d8cc-484c-a30c-be92c9776a04-host-slash\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158389 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysctl-d\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158402 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-run\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158416 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158447 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158491 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-systemd\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158516 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-ovnkube-script-lib\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158549 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-daemon-config\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158584 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-registration-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158614 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-etc-kubernetes\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.159293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158636 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-tmp-dir\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.159947 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.158659 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-ovn\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.198800 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.198777 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 00:10:39.218669 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.218637 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t5sw8" Jun 09 00:10:39.224459 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.224422 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t5sw8" Jun 09 00:10:39.248273 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.248237 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 00:10:39.258985 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.258948 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-k8s-cni-cncf-io\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.258985 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.258991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qjrc\" (UniqueName: \"kubernetes.io/projected/e7ba130d-8a9d-4fc7-bc44-009b00be3655-kube-api-access-8qjrc\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259010 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysctl-conf\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259033 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-sys\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259072 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-k8s-cni-cncf-io\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259123 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52g96\" (UniqueName: \"kubernetes.io/projected/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-kube-api-access-52g96\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259144 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-run-netns\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259168 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259193 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-node-log\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259217 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-log-socket\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259223 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-run-netns\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259240 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259228 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysctl-conf\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.259254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259256 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-ovnkube-config\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259295 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-multus-certs\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259302 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-node-log\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259319 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-hosts-file\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259342 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-cni-multus\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259344 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-log-socket\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259363 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec8d62a2-d8cc-484c-a30c-be92c9776a04-host-slash\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259385 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysctl-d\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259388 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-hosts-file\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259407 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-run\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259425 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-sys\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259424 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-multus-certs\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259447 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259462 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec8d62a2-d8cc-484c-a30c-be92c9776a04-host-slash\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259488 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259516 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-systemd\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259536 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-ovnkube-script-lib\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259541 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-cni-multus\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.259738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259555 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-daemon-config\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.260499 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259568 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-systemd\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.260499 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259581 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-run\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.260499 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259575 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-registration-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.260499 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259606 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysctl-d\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.260499 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259615 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-etc-kubernetes\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.260499 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.259667 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:39.260959 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.260947 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:10:39.760917171 +0000 UTC m=+2.985314435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:39.261033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.260633 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-ovnkube-script-lib\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.261033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.260651 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-tmp-dir\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.261033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261020 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-ovn\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.261175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261056 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.261175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261108 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-kubelet\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.261175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261135 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-hostroot\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.261175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261169 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.261330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7cp\" (UniqueName: \"kubernetes.io/projected/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-kube-api-access-fz7cp\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.261330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261216 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-serviceca\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.261330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261263 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f62773d3-957a-4951-8974-a26ee5ebd3cb-kubelet-config\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.261330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261297 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-systemd-units\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.261330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261326 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-var-lib-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.261503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261359 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-cni-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.261503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261389 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-conf-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.261503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261415 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-sys-fs\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.261503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261436 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp6r8\" (UniqueName: \"kubernetes.io/projected/ec8d62a2-d8cc-484c-a30c-be92c9776a04-kube-api-access-sp6r8\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.261503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261452 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-modprobe-d\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.261503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261476 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv6wm\" (UniqueName: \"kubernetes.io/projected/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-kube-api-access-jv6wm\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261512 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysconfig\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261535 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-kubernetes\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261561 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/546188fe-9f2f-4ea9-8775-39af993e771d-etc-tuned\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261582 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-system-cni-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261611 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-os-release\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261642 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-system-cni-dir\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261668 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-host\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.261726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261706 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-slash\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261735 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-run-ovn-kubernetes\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261755 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-cni-bin\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261775 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261803 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b286762b-0f99-494b-9e7c-d89fbd551c00-cni-binary-copy\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-netns\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261864 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-etc-selinux\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261894 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfccj\" (UniqueName: \"kubernetes.io/projected/dd74db37-2abc-4b24-8362-f410ec8c144a-kube-api-access-bfccj\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261921 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261946 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f62773d3-957a-4951-8974-a26ee5ebd3cb-dbus\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261970 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ec8d62a2-d8cc-484c-a30c-be92c9776a04-iptables-alerter-script\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.261991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-lib-modules\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262011 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-var-lib-kubelet\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262030 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b633971-bf33-4527-b021-b8f97a6b1f69-agent-certs\") pod \"konnectivity-agent-jzrmb\" (UID: \"4b633971-bf33-4527-b021-b8f97a6b1f69\") " pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.262060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262062 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-cnibin\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262105 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-os-release\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262140 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262170 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/546188fe-9f2f-4ea9-8775-39af993e771d-tmp\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262193 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-cni-netd\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262214 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-cni-bin\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262237 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcv96\" (UniqueName: \"kubernetes.io/projected/b286762b-0f99-494b-9e7c-d89fbd551c00-kube-api-access-tcv96\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262257 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-socket-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262278 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-device-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262302 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-host\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262328 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-etc-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262363 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-env-overrides\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262396 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262443 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5wx\" (UniqueName: \"kubernetes.io/projected/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-kube-api-access-hx5wx\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262464 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk57k\" (UniqueName: \"kubernetes.io/projected/546188fe-9f2f-4ea9-8775-39af993e771d-kube-api-access-zk57k\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262481 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-kubelet\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262508 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd74db37-2abc-4b24-8362-f410ec8c144a-ovn-node-metrics-cert\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.262582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262531 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cnibin\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262554 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-systemd\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262577 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b633971-bf33-4527-b021-b8f97a6b1f69-konnectivity-ca\") pod \"konnectivity-agent-jzrmb\" (UID: \"4b633971-bf33-4527-b021-b8f97a6b1f69\") " pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262605 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-socket-dir-parent\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262710 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-socket-dir-parent\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.260198 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-ovnkube-config\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.260868 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-tmp-dir\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.262995 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-run-ovn\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259678 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-etc-kubernetes\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.263041 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.263095 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-kubelet\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.263322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.263142 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-hostroot\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.263752 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.263595 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.264228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264209 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-serviceca\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.264319 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264295 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f62773d3-957a-4951-8974-a26ee5ebd3cb-kubelet-config\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.264370 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264348 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-systemd-units\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.264405 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264391 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-var-lib-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.264455 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264434 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-cni-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.264501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264474 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-conf-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.264543 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264528 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-sys-fs\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.264828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.264811 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-modprobe-d\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.265147 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.265132 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-sysconfig\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.265227 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.265195 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-kubernetes\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.265514 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.265493 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.259517 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.265928 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-host\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.265967 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266001 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-cnibin\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.266100 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:39.766051964 +0000 UTC m=+2.990449251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266103 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-system-cni-dir\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266191 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-os-release\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266267 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-system-cni-dir\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266293 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-slash\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266316 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-cni-bin\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.266576 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266205 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-run-ovn-kubernetes\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.267165 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.265841 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-registration-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.267165 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266617 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.267165 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.266913 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-run-netns\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.267165 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267009 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-etc-selinux\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.267360 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267191 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-etc-openvswitch\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.267360 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267263 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b286762b-0f99-494b-9e7c-d89fbd551c00-cni-binary-copy\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.267360 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267310 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-os-release\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.267360 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267310 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-lib-modules\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.267557 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267361 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b286762b-0f99-494b-9e7c-d89fbd551c00-multus-daemon-config\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.267557 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267363 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-kubelet\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.267557 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267417 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f62773d3-957a-4951-8974-a26ee5ebd3cb-dbus\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.267701 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267611 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b286762b-0f99-494b-9e7c-d89fbd551c00-host-var-lib-cni-bin\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.267701 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267615 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cnibin\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.267801 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267753 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ec8d62a2-d8cc-484c-a30c-be92c9776a04-iptables-alerter-script\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.267858 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267806 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dd74db37-2abc-4b24-8362-f410ec8c144a-env-overrides\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.267906 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267853 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-etc-systemd\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.267962 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267920 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/546188fe-9f2f-4ea9-8775-39af993e771d-var-lib-kubelet\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.267962 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.267939 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.268071 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.268014 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-host\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.268167 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.268120 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-device-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.268224 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.268168 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7ba130d-8a9d-4fc7-bc44-009b00be3655-socket-dir\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.268289 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.268270 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b633971-bf33-4527-b021-b8f97a6b1f69-konnectivity-ca\") pod \"konnectivity-agent-jzrmb\" (UID: \"4b633971-bf33-4527-b021-b8f97a6b1f69\") " pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.268344 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.268299 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.268396 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.268341 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dd74db37-2abc-4b24-8362-f410ec8c144a-host-cni-netd\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.269904 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.269881 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/546188fe-9f2f-4ea9-8775-39af993e771d-etc-tuned\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.271210 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.271189 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/546188fe-9f2f-4ea9-8775-39af993e771d-tmp\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.271651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.271626 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qjrc\" (UniqueName: \"kubernetes.io/projected/e7ba130d-8a9d-4fc7-bc44-009b00be3655-kube-api-access-8qjrc\") pod \"aws-ebs-csi-driver-node-58jt7\" (UID: \"e7ba130d-8a9d-4fc7-bc44-009b00be3655\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.271772 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.271747 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b633971-bf33-4527-b021-b8f97a6b1f69-agent-certs\") pod \"konnectivity-agent-jzrmb\" (UID: \"4b633971-bf33-4527-b021-b8f97a6b1f69\") " pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.271837 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.271772 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:10:39.271837 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.271798 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:10:39.271837 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.271812 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:39.271837 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.271751 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd74db37-2abc-4b24-8362-f410ec8c144a-ovn-node-metrics-cert\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.272046 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.271897 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:39.771871049 +0000 UTC m=+2.996268298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:39.272184 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.272163 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52g96\" (UniqueName: \"kubernetes.io/projected/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-kube-api-access-52g96\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.274352 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.274331 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7cp\" (UniqueName: \"kubernetes.io/projected/ccd3a06c-34ce-446e-8593-860e6a8eb7c6-kube-api-access-fz7cp\") pod \"multus-additional-cni-plugins-rt7rv\" (UID: \"ccd3a06c-34ce-446e-8593-860e6a8eb7c6\") " pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.274713 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.274650 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv6wm\" (UniqueName: \"kubernetes.io/projected/3e0a6921-f28b-42f1-a2c9-05ed1e65c4de-kube-api-access-jv6wm\") pod \"node-ca-nj5z7\" (UID: \"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de\") " pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.275550 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.275469 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfccj\" (UniqueName: \"kubernetes.io/projected/dd74db37-2abc-4b24-8362-f410ec8c144a-kube-api-access-bfccj\") pod \"ovnkube-node-wks97\" (UID: \"dd74db37-2abc-4b24-8362-f410ec8c144a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.275639 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.275570 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp6r8\" (UniqueName: \"kubernetes.io/projected/ec8d62a2-d8cc-484c-a30c-be92c9776a04-kube-api-access-sp6r8\") pod \"iptables-alerter-4dqc4\" (UID: \"ec8d62a2-d8cc-484c-a30c-be92c9776a04\") " pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.276868 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.276846 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk57k\" (UniqueName: \"kubernetes.io/projected/546188fe-9f2f-4ea9-8775-39af993e771d-kube-api-access-zk57k\") pod \"tuned-h6krf\" (UID: \"546188fe-9f2f-4ea9-8775-39af993e771d\") " pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.276955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.276921 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcv96\" (UniqueName: \"kubernetes.io/projected/b286762b-0f99-494b-9e7c-d89fbd551c00-kube-api-access-tcv96\") pod \"multus-g9lwf\" (UID: \"b286762b-0f99-494b-9e7c-d89fbd551c00\") " pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.277227 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.277209 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5wx\" (UniqueName: \"kubernetes.io/projected/dc81db3e-2bb4-42bc-9b8d-c07d09d3226e-kube-api-access-hx5wx\") pod \"node-resolver-g97wc\" (UID: \"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e\") " pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.325323 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.325291 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 00:10:39.441973 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.441880 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" Jun 09 00:10:39.450605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.450576 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h6krf" Jun 09 00:10:39.458380 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.458360 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:10:39.464220 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.464185 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" Jun 09 00:10:39.471780 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.471755 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nj5z7" Jun 09 00:10:39.478456 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.478433 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:10:39.485500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.485478 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9lwf" Jun 09 00:10:39.493153 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.493129 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dqc4" Jun 09 00:10:39.497759 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.497737 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g97wc" Jun 09 00:10:39.766824 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.766734 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:39.766824 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.766787 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:39.767041 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.766905 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:39.767041 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.766962 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:40.766945356 +0000 UTC m=+3.991342623 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:39.767041 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.766990 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:39.767218 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.767060 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:10:40.767041733 +0000 UTC m=+3.991438985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:39.867981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:39.867947 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:39.868164 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.868146 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:10:39.868251 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.868172 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:10:39.868251 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.868189 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:39.868338 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:39.868260 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:40.86824401 +0000 UTC m=+4.092641256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:40.039361 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.039333 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8d62a2_d8cc_484c_a30c_be92c9776a04.slice/crio-f5615075cde80327acab7b67017f052fd676609d4cc992297ee070535358b90d WatchSource:0}: Error finding container f5615075cde80327acab7b67017f052fd676609d4cc992297ee070535358b90d: Status 404 returned error can't find the container with id f5615075cde80327acab7b67017f052fd676609d4cc992297ee070535358b90d Jun 09 00:10:40.040926 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.040885 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd3a06c_34ce_446e_8593_860e6a8eb7c6.slice/crio-863151fa73d7ab83c9ab11fe5943d6881de550492708491629fbf060b18b6950 WatchSource:0}: Error finding container 863151fa73d7ab83c9ab11fe5943d6881de550492708491629fbf060b18b6950: Status 404 returned error can't find the container with id 863151fa73d7ab83c9ab11fe5943d6881de550492708491629fbf060b18b6950 Jun 09 00:10:40.041471 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.041435 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb286762b_0f99_494b_9e7c_d89fbd551c00.slice/crio-2e1686816abd106f52019a4181df826434c128e807e70aa188a2bfc3568add55 WatchSource:0}: Error finding container 2e1686816abd106f52019a4181df826434c128e807e70aa188a2bfc3568add55: Status 404 returned error can't find the container with id 2e1686816abd106f52019a4181df826434c128e807e70aa188a2bfc3568add55 Jun 09 00:10:40.044913 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.044893 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546188fe_9f2f_4ea9_8775_39af993e771d.slice/crio-b8b9657073f0ebcc6ee8f5af286839db36bdfb68627ed5ff467c710b664d669f WatchSource:0}: Error finding container b8b9657073f0ebcc6ee8f5af286839db36bdfb68627ed5ff467c710b664d669f: Status 404 returned error can't find the container with id b8b9657073f0ebcc6ee8f5af286839db36bdfb68627ed5ff467c710b664d669f Jun 09 00:10:40.048273 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.048193 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7ba130d_8a9d_4fc7_bc44_009b00be3655.slice/crio-35e5010c7c0c3a30d7d917bf2426a4a98f43a7aacaa55b41f6107eb569d660bc WatchSource:0}: Error finding container 35e5010c7c0c3a30d7d917bf2426a4a98f43a7aacaa55b41f6107eb569d660bc: Status 404 returned error can't find the container with id 35e5010c7c0c3a30d7d917bf2426a4a98f43a7aacaa55b41f6107eb569d660bc Jun 09 00:10:40.049231 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.048403 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b633971_bf33_4527_b021_b8f97a6b1f69.slice/crio-4d5de559b92a4c027bc64973fa1b28dfa5c01a7eed3f1e227648ee0c4ea2287c WatchSource:0}: Error finding container 4d5de559b92a4c027bc64973fa1b28dfa5c01a7eed3f1e227648ee0c4ea2287c: Status 404 returned error can't find the container with id 4d5de559b92a4c027bc64973fa1b28dfa5c01a7eed3f1e227648ee0c4ea2287c Jun 09 00:10:40.049231 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:10:40.049113 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd74db37_2abc_4b24_8362_f410ec8c144a.slice/crio-1b0ecd47ebb5be2022e524f8e7278f58f1c61ea93753bcd3b1f19b97bdfaf216 WatchSource:0}: Error finding container 1b0ecd47ebb5be2022e524f8e7278f58f1c61ea93753bcd3b1f19b97bdfaf216: Status 404 returned error can't find the container with id 1b0ecd47ebb5be2022e524f8e7278f58f1c61ea93753bcd3b1f19b97bdfaf216 Jun 09 00:10:40.226597 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.226396 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 00:05:39 +0000 UTC" deadline="2028-01-09 09:32:43.597434825 +0000 UTC" Jun 09 00:10:40.227061 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.226619 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13905h22m3.370822424s" Jun 09 00:10:40.238399 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.238361 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jzrmb" event={"ID":"4b633971-bf33-4527-b021-b8f97a6b1f69","Type":"ContainerStarted","Data":"4d5de559b92a4c027bc64973fa1b28dfa5c01a7eed3f1e227648ee0c4ea2287c"} Jun 09 00:10:40.240556 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.240483 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" event={"ID":"b2ca2fd9e0ea5addc9b16951a720ff81","Type":"ContainerStarted","Data":"87ba15c37bed726d5fa76702e4aac77f895c8dfa084787325d813b6e868766d4"} Jun 09 00:10:40.242036 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.242011 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g97wc" event={"ID":"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e","Type":"ContainerStarted","Data":"e9aac918a5fbe58f383fe08bf9074a112f82d726fa17fc27930b629923d48714"} Jun 09 00:10:40.243314 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.243289 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"1b0ecd47ebb5be2022e524f8e7278f58f1c61ea93753bcd3b1f19b97bdfaf216"} Jun 09 00:10:40.245275 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.245255 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" event={"ID":"e7ba130d-8a9d-4fc7-bc44-009b00be3655","Type":"ContainerStarted","Data":"35e5010c7c0c3a30d7d917bf2426a4a98f43a7aacaa55b41f6107eb569d660bc"} Jun 09 00:10:40.246447 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.246426 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h6krf" event={"ID":"546188fe-9f2f-4ea9-8775-39af993e771d","Type":"ContainerStarted","Data":"b8b9657073f0ebcc6ee8f5af286839db36bdfb68627ed5ff467c710b664d669f"} Jun 09 00:10:40.247590 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.247567 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9lwf" event={"ID":"b286762b-0f99-494b-9e7c-d89fbd551c00","Type":"ContainerStarted","Data":"2e1686816abd106f52019a4181df826434c128e807e70aa188a2bfc3568add55"} Jun 09 00:10:40.248644 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.248627 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerStarted","Data":"863151fa73d7ab83c9ab11fe5943d6881de550492708491629fbf060b18b6950"} Jun 09 00:10:40.249902 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.249883 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dqc4" event={"ID":"ec8d62a2-d8cc-484c-a30c-be92c9776a04","Type":"ContainerStarted","Data":"f5615075cde80327acab7b67017f052fd676609d4cc992297ee070535358b90d"} Jun 09 00:10:40.251096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.251063 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nj5z7" event={"ID":"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de","Type":"ContainerStarted","Data":"c6da4062bb809224f1391432e49a821950794d1960a9a90f80f06403c5baf20d"} Jun 09 00:10:40.253553 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.253516 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-86.ec2.internal" podStartSLOduration=2.253503089 podStartE2EDuration="2.253503089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:40.253401845 +0000 UTC m=+3.477799111" watchObservedRunningTime="2026-06-09 00:10:40.253503089 +0000 UTC m=+3.477900358" Jun 09 00:10:40.777468 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.776658 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:40.777468 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.776750 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:40.777468 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.776873 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:40.777468 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.776935 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:10:42.776915633 +0000 UTC m=+6.001312883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:40.777468 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.777003 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:40.777468 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.777040 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:42.777027244 +0000 UTC m=+6.001424508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:40.877511 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:40.877472 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:40.877678 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.877664 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:10:40.877753 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.877682 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:10:40.877753 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.877695 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:40.877851 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:40.877758 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:42.877738683 +0000 UTC m=+6.102135943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:41.227108 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.226911 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 00:05:39 +0000 UTC" deadline="2028-01-29 03:36:00.201831701 +0000 UTC" Jun 09 00:10:41.227108 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.226973 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14379h25m18.974863521s" Jun 09 00:10:41.232947 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.232918 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:41.233115 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:41.233044 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:41.233492 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.233471 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:41.233609 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:41.233572 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:41.233669 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.233638 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:41.233728 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:41.233706 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:41.289198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.288421 2550 generic.go:356] "Generic (PLEG): container finished" podID="8ea97a5bdfd910401bb9da830322516f" containerID="ecbde314c4b2d2e6bb2383394b7916547e8cc42e6d851fb0f5524754a82c08cb" exitCode=0 Jun 09 00:10:41.289198 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:41.289143 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" event={"ID":"8ea97a5bdfd910401bb9da830322516f","Type":"ContainerDied","Data":"ecbde314c4b2d2e6bb2383394b7916547e8cc42e6d851fb0f5524754a82c08cb"} Jun 09 00:10:42.300720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:42.300684 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" event={"ID":"8ea97a5bdfd910401bb9da830322516f","Type":"ContainerStarted","Data":"f4d809bf6f3aeef6420681515737ce34af34879eef91dd6d6e3fa14f5fc791ed"} Jun 09 00:10:42.797958 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:42.797919 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:42.798184 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:42.797990 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:42.798184 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.798171 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:42.798303 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.798235 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:46.798215674 +0000 UTC m=+10.022612932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:42.798646 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.798616 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:42.798737 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.798671 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:10:46.798657445 +0000 UTC m=+10.023054696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:42.898862 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:42.898825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:42.899095 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.899063 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:10:42.899170 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.899106 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:10:42.899170 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.899119 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:42.899298 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:42.899173 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:46.899159777 +0000 UTC m=+10.123557023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:43.230558 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:43.230155 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:43.230558 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:43.230309 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:43.230797 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:43.230666 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:43.230797 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:43.230754 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:43.230901 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:43.230801 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:43.230901 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:43.230867 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:45.229651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:45.229609 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:45.230124 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:45.229626 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:45.230124 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:45.229754 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:45.230124 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:45.229838 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:45.230124 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:45.229843 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:45.230124 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:45.229907 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:46.832849 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:46.832809 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:46.833434 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:46.832876 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:46.833434 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.833001 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:46.833434 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.833015 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:46.833434 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.833069 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:10:54.833055319 +0000 UTC m=+18.057452568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:46.833434 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.833111 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:54.833097884 +0000 UTC m=+18.057495140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:46.933407 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:46.933368 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:46.933565 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.933507 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:10:46.933565 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.933521 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:10:46.933565 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.933530 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:46.933670 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:46.933579 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:10:54.933566793 +0000 UTC m=+18.157964039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:47.234368 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:47.234336 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:47.234545 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:47.234334 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:47.234545 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:47.234457 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:47.234545 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:47.234336 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:47.234545 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:47.234526 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:47.234782 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:47.234631 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:49.229502 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:49.229462 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:49.230067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:49.229462 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:49.230067 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:49.229605 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:49.230067 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:49.229680 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:49.230067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:49.229735 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:49.230067 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:49.229786 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:51.230014 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:51.229518 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:51.230014 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:51.229535 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:51.230014 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:51.229666 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:51.230014 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:51.229724 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:51.230014 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:51.229769 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:51.230014 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:51.229842 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:53.230057 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:53.230007 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:53.230057 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:53.230059 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:53.230722 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:53.230065 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:53.230722 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:53.230177 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:53.230722 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:53.230241 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:53.230722 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:53.230385 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:54.894162 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:54.894127 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:54.894721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:54.894195 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:54.894721 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.894269 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:54.894721 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.894306 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:54.894721 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.894345 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:11:10.894322607 +0000 UTC m=+34.118719870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:10:54.894721 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.894366 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:10.89435411 +0000 UTC m=+34.118751357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:10:54.994868 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:54.994822 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:54.995055 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.995000 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:10:54.995055 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.995021 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:10:54.995055 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.995031 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:54.995259 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:54.995106 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:10.99507232 +0000 UTC m=+34.219469581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:10:55.229629 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:55.229590 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:55.229629 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:55.229626 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:55.229876 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:55.229590 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:55.229876 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:55.229724 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:55.229876 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:55.229802 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:55.229876 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:55.229856 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:57.230437 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:57.230404 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:57.230890 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:57.230498 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:57.230890 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:57.230547 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:57.230890 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:57.230597 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:57.230890 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:57.230626 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:57.230890 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:57.230738 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:59.229911 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.229881 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:10:59.230526 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:59.230009 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:10:59.230526 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.229881 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:10:59.230526 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:59.230236 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:10:59.230526 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.230467 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:10:59.230700 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:10:59.230553 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:10:59.334678 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.334646 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g97wc" event={"ID":"dc81db3e-2bb4-42bc-9b8d-c07d09d3226e","Type":"ContainerStarted","Data":"39f2045cb0d40f826fe3544e15fc636ae43f0c7130414333d596738c2ecac2f5"} Jun 09 00:10:59.336886 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.336857 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"523e8a6671b5ba401263aca20c23f23137976dd35aae52e5bfea96721754c411"} Jun 09 00:10:59.336991 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.336943 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"08188d0d41c0b4a3c736eb76970d0c955bb2f9190557489627fbc2aed5f570f2"} Jun 09 00:10:59.338541 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.338515 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" event={"ID":"e7ba130d-8a9d-4fc7-bc44-009b00be3655","Type":"ContainerStarted","Data":"4e931cea102ddfbdd627921e57d931931c0dbcae74903ac76ffaeab0c15b4b3c"} Jun 09 00:10:59.340123 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.340056 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h6krf" event={"ID":"546188fe-9f2f-4ea9-8775-39af993e771d","Type":"ContainerStarted","Data":"b054214bdd4c3f8782e693517245dac4cc7d0171f763e7001096df20c8b3d9b6"} Jun 09 00:10:59.341615 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.341595 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9lwf" event={"ID":"b286762b-0f99-494b-9e7c-d89fbd551c00","Type":"ContainerStarted","Data":"553fb0e601522e2d44d09f439fb9f35b6f94a1ebf55fa227119e0309c936f3d4"} Jun 09 00:10:59.343035 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.343014 2550 generic.go:356] "Generic (PLEG): container finished" podID="ccd3a06c-34ce-446e-8593-860e6a8eb7c6" containerID="e66695ff30b32f222564f95c10819adb65a8d8ae6cc88547b07aebf5c6c0520b" exitCode=0 Jun 09 00:10:59.343145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.343115 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerDied","Data":"e66695ff30b32f222564f95c10819adb65a8d8ae6cc88547b07aebf5c6c0520b"} Jun 09 00:10:59.344542 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.344511 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nj5z7" event={"ID":"3e0a6921-f28b-42f1-a2c9-05ed1e65c4de","Type":"ContainerStarted","Data":"b07addf4ceec3fce9c42609bf5e9431272d356159f06497452ed185152055111"} Jun 09 00:10:59.345976 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.345958 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jzrmb" event={"ID":"4b633971-bf33-4527-b021-b8f97a6b1f69","Type":"ContainerStarted","Data":"959f8bd02cdee3127bf629f74378bb15f6c027e3fb5a5e72c5110810fd90aa90"} Jun 09 00:10:59.349794 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.349753 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g97wc" podStartSLOduration=3.483131051 podStartE2EDuration="22.349741636s" totalImagesPullingTime="18.866610585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:59.349500946 +0000 UTC m=+22.573898216" watchObservedRunningTime="2026-06-09 00:10:59.349741636 +0000 UTC m=+22.574138904" Jun 09 00:10:59.349904 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.349820 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-86.ec2.internal" podStartSLOduration=21.349815447 podStartE2EDuration="21.349815447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:42.315834797 +0000 UTC m=+5.540232066" watchObservedRunningTime="2026-06-09 00:10:59.349815447 +0000 UTC m=+22.574212715" Jun 09 00:10:59.371363 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.371312 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jzrmb" podStartSLOduration=8.171247746 podStartE2EDuration="22.37129686s" totalImagesPullingTime="14.200049114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:59.371156101 +0000 UTC m=+22.595553369" watchObservedRunningTime="2026-06-09 00:10:59.37129686 +0000 UTC m=+22.595694128" Jun 09 00:10:59.427331 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.427250 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h6krf" podStartSLOduration=3.553835434 podStartE2EDuration="22.427233673s" totalImagesPullingTime="18.873398239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:59.427007771 +0000 UTC m=+22.651405039" watchObservedRunningTime="2026-06-09 00:10:59.427233673 +0000 UTC m=+22.651630919" Jun 09 00:10:59.427577 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.427549 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g9lwf" podStartSLOduration=3.504566639 podStartE2EDuration="22.427539378s" totalImagesPullingTime="18.922972739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:59.412510563 +0000 UTC m=+22.636907831" watchObservedRunningTime="2026-06-09 00:10:59.427539378 +0000 UTC m=+22.651936645" Jun 09 00:10:59.441430 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:10:59.441258 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nj5z7" podStartSLOduration=3.57542648 podStartE2EDuration="22.441243195s" totalImagesPullingTime="18.865816715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:10:59.441009794 +0000 UTC m=+22.665407123" watchObservedRunningTime="2026-06-09 00:10:59.441243195 +0000 UTC m=+22.665640463" Jun 09 00:11:00.059196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.059170 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 00:11:00.222256 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.222142 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T00:11:00.059191236Z","UUID":"3951cc3f-fb89-4921-a71f-13baf675621b","Handler":null,"Name":"","Endpoint":""} Jun 09 00:11:00.223622 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.223595 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 00:11:00.223751 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.223641 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 00:11:00.352728 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.352691 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"fd15037383adac31d43f1c253dee55608007ffe66aa1868e3e15b94b483d10cf"} Jun 09 00:11:00.353312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.352820 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"dac99146b0d01242cdc54188877c254509c531ed7a0fa8a0b8552496eadc6454"} Jun 09 00:11:00.353312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.352874 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"bdb266e1aa96173a7c3db27c58c318b171d1cc6f155a96e2eff9f6f6d9f5f428"} Jun 09 00:11:00.353312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.352927 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"ac050e764b9fd1d6fd59c5bb3d07aff2f1413788553ed00d2a4df5f1e45a48dd"} Jun 09 00:11:00.355068 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:00.355037 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" event={"ID":"e7ba130d-8a9d-4fc7-bc44-009b00be3655","Type":"ContainerStarted","Data":"8732b993acc878a555031b22a2538c5d8397177fa2e09cd8132e085205d34cc6"} Jun 09 00:11:01.233040 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.233006 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:01.233226 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.233009 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:01.233226 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:01.233132 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:11:01.233226 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.233166 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:01.233377 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:01.233274 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:11:01.233377 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:01.233357 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:11:01.359462 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.359428 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" event={"ID":"e7ba130d-8a9d-4fc7-bc44-009b00be3655","Type":"ContainerStarted","Data":"dbc6d7950db3cf367dd971cc2a828d736887cdecde1eaa69b1738e8c951ae3d0"} Jun 09 00:11:01.361071 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.361028 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dqc4" event={"ID":"ec8d62a2-d8cc-484c-a30c-be92c9776a04","Type":"ContainerStarted","Data":"825374e697ecb5ba180df5758067d195dc87ea1743472cf608eb9bfebf3f54de"} Jun 09 00:11:01.380246 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.380169 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58jt7" podStartSLOduration=12.521083439 podStartE2EDuration="24.380150575s" totalImagesPullingTime="11.859067136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:01.380123046 +0000 UTC m=+24.604520313" watchObservedRunningTime="2026-06-09 00:11:01.380150575 +0000 UTC m=+24.604547824" Jun 09 00:11:01.395301 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.395238 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4dqc4" podStartSLOduration=5.515507127 podStartE2EDuration="24.395221086s" totalImagesPullingTime="18.879713959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:01.394938518 +0000 UTC m=+24.619335786" watchObservedRunningTime="2026-06-09 00:11:01.395221086 +0000 UTC m=+24.619618378" Jun 09 00:11:01.575663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.575573 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:11:01.576243 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:01.576220 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:11:02.367111 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:02.367058 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"ffa698ae41e77fe011cb2a7b83b042449a0e883b6e0cb693c3e57a6b63aa75ae"} Jun 09 00:11:02.367601 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:02.367388 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:11:02.367783 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:02.367768 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jzrmb" Jun 09 00:11:03.233141 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:03.233111 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:03.233331 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:03.233151 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:03.233331 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:03.233111 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:03.233331 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:03.233237 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:11:03.233331 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:03.233295 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:11:03.233558 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:03.233360 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:11:05.229803 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.229616 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:05.230463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.229628 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:05.230463 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:05.229884 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:11:05.230463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.229628 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:05.230463 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:05.229947 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:11:05.230463 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:05.230006 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:11:05.376330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.376276 2550 generic.go:356] "Generic (PLEG): container finished" podID="ccd3a06c-34ce-446e-8593-860e6a8eb7c6" containerID="ad2cf2d77a229f5df5a0d1c07ec153dcf9e6d6d3f3c2ad4569146e82e6ccd659" exitCode=0 Jun 09 00:11:05.376330 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.376324 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerDied","Data":"ad2cf2d77a229f5df5a0d1c07ec153dcf9e6d6d3f3c2ad4569146e82e6ccd659"} Jun 09 00:11:05.380139 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.380063 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" event={"ID":"dd74db37-2abc-4b24-8362-f410ec8c144a","Type":"ContainerStarted","Data":"4bda9214443a8df22f4a1efb166344721d43ab3f1b1ed7122def48900c8cf61b"} Jun 09 00:11:05.380366 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.380349 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:11:05.380445 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.380428 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:11:05.396904 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.396875 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:11:05.429562 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:05.429452 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" podStartSLOduration=9.464307423 podStartE2EDuration="28.429437462s" totalImagesPullingTime="18.965130039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:05.429061325 +0000 UTC m=+28.653458592" watchObservedRunningTime="2026-06-09 00:11:05.429437462 +0000 UTC m=+28.653834754" Jun 09 00:11:06.382666 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.382427 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:11:06.398170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.398145 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:11:06.927874 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.927796 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbvbz"] Jun 09 00:11:06.928009 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.927911 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:06.928009 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:06.928000 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:11:06.931281 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.931242 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x8h82"] Jun 09 00:11:06.931434 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.931379 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:06.931521 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:06.931491 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:11:06.931911 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.931884 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vx6qc"] Jun 09 00:11:06.932019 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:06.932004 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:06.932154 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:06.932132 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:11:07.385847 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:07.385812 2550 generic.go:356] "Generic (PLEG): container finished" podID="ccd3a06c-34ce-446e-8593-860e6a8eb7c6" containerID="021e6dceef173a63eb1b2cded2dc04e6a1b624c2c08a0b41d1cc514e40b11285" exitCode=0 Jun 09 00:11:07.386529 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:07.385895 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerDied","Data":"021e6dceef173a63eb1b2cded2dc04e6a1b624c2c08a0b41d1cc514e40b11285"} Jun 09 00:11:08.229760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:08.229729 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:08.229917 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:08.229729 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:08.229917 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:08.229844 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:11:08.229987 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:08.229910 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:11:09.233276 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:09.233242 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:09.233861 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:09.233368 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:11:09.391993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:09.391963 2550 generic.go:356] "Generic (PLEG): container finished" podID="ccd3a06c-34ce-446e-8593-860e6a8eb7c6" containerID="c68831e290355e35a9621960ce04da41979e26bc64d86299446f9b259f8aaea7" exitCode=0 Jun 09 00:11:09.392188 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:09.392009 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerDied","Data":"c68831e290355e35a9621960ce04da41979e26bc64d86299446f9b259f8aaea7"} Jun 09 00:11:10.230048 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:10.229833 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:10.230260 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:10.229879 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:10.230260 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:10.230175 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x8h82" podUID="f62773d3-957a-4951-8974-a26ee5ebd3cb" Jun 09 00:11:10.230362 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:10.230343 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbvbz" podUID="9df01a83-e004-4e3f-aa1f-8a928873c214" Jun 09 00:11:10.907879 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:10.907838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:10.908399 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:10.907930 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:10.908399 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:10.908011 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:11:10.908399 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:10.908044 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 00:11:10.908399 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:10.908104 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:42.908066544 +0000 UTC m=+66.132463790 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 00:11:10.908399 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:10.908127 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret podName:f62773d3-957a-4951-8974-a26ee5ebd3cb nodeName:}" failed. No retries permitted until 2026-06-09 00:11:42.908116678 +0000 UTC m=+66.132513929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret") pod "global-pull-secret-syncer-x8h82" (UID: "f62773d3-957a-4951-8974-a26ee5ebd3cb") : object "kube-system"/"original-pull-secret" not registered Jun 09 00:11:11.008336 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.008293 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:11.008505 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.008480 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 00:11:11.008574 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.008509 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 00:11:11.008574 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.008521 2550 projected.go:196] Error preparing data for projected volume kube-api-access-8w8jg for pod openshift-network-diagnostics/network-check-target-jbvbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:11:11.008574 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.008573 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg podName:9df01a83-e004-4e3f-aa1f-8a928873c214 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:43.008558961 +0000 UTC m=+66.232956207 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8w8jg" (UniqueName: "kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg") pod "network-check-target-jbvbz" (UID: "9df01a83-e004-4e3f-aa1f-8a928873c214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 00:11:11.233405 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.233381 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:11.233569 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.233499 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:11:11.638867 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.638832 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-86.ec2.internal" event="NodeReady" Jun 09 00:11:11.639036 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.638990 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 00:11:11.681267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.681226 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pd9ph"] Jun 09 00:11:11.693261 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.693226 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gtn7g"] Jun 09 00:11:11.693439 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.693395 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.697609 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.696199 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hv7fp\"" Jun 09 00:11:11.697609 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.696781 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 00:11:11.697609 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.696795 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 00:11:11.703744 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.703687 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pd9ph"] Jun 09 00:11:11.703744 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.703731 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gtn7g"] Jun 09 00:11:11.703915 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.703845 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.706363 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.706341 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 00:11:11.706476 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.706405 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n9dzh\"" Jun 09 00:11:11.706476 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.706342 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 00:11:11.706640 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.706622 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 00:11:11.814359 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.814241 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bf29450-7699-4d9e-8505-87f6b8688f9c-cert\") pod \"ingress-canary-gtn7g\" (UID: \"9bf29450-7699-4d9e-8505-87f6b8688f9c\") " pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.814359 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.814283 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/541a23f9-36b8-4b1f-a7a3-254f348ec588-config-volume\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.814359 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.814341 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.814632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.814430 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541a23f9-36b8-4b1f-a7a3-254f348ec588-tmp-dir\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.814632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.814451 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwrs5\" (UniqueName: \"kubernetes.io/projected/9bf29450-7699-4d9e-8505-87f6b8688f9c-kube-api-access-jwrs5\") pod \"ingress-canary-gtn7g\" (UID: \"9bf29450-7699-4d9e-8505-87f6b8688f9c\") " pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.814632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.814524 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjt4k\" (UniqueName: \"kubernetes.io/projected/541a23f9-36b8-4b1f-a7a3-254f348ec588-kube-api-access-gjt4k\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.915515 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.915478 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bf29450-7699-4d9e-8505-87f6b8688f9c-cert\") pod \"ingress-canary-gtn7g\" (UID: \"9bf29450-7699-4d9e-8505-87f6b8688f9c\") " pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.915515 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.915528 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/541a23f9-36b8-4b1f-a7a3-254f348ec588-config-volume\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.916032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.915673 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.916032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.915767 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541a23f9-36b8-4b1f-a7a3-254f348ec588-tmp-dir\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.916032 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.915794 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:11.916032 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:11.915866 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:12.415843618 +0000 UTC m=+35.640240882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:11.916032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.915797 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwrs5\" (UniqueName: \"kubernetes.io/projected/9bf29450-7699-4d9e-8505-87f6b8688f9c-kube-api-access-jwrs5\") pod \"ingress-canary-gtn7g\" (UID: \"9bf29450-7699-4d9e-8505-87f6b8688f9c\") " pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.916032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.915949 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjt4k\" (UniqueName: \"kubernetes.io/projected/541a23f9-36b8-4b1f-a7a3-254f348ec588-kube-api-access-gjt4k\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.916376 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.916037 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541a23f9-36b8-4b1f-a7a3-254f348ec588-tmp-dir\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.916376 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.916155 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/541a23f9-36b8-4b1f-a7a3-254f348ec588-config-volume\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:11.920758 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.920732 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bf29450-7699-4d9e-8505-87f6b8688f9c-cert\") pod \"ingress-canary-gtn7g\" (UID: \"9bf29450-7699-4d9e-8505-87f6b8688f9c\") " pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.926979 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.926949 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwrs5\" (UniqueName: \"kubernetes.io/projected/9bf29450-7699-4d9e-8505-87f6b8688f9c-kube-api-access-jwrs5\") pod \"ingress-canary-gtn7g\" (UID: \"9bf29450-7699-4d9e-8505-87f6b8688f9c\") " pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:11.929494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:11.929468 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjt4k\" (UniqueName: \"kubernetes.io/projected/541a23f9-36b8-4b1f-a7a3-254f348ec588-kube-api-access-gjt4k\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:12.016734 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.016696 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtn7g" Jun 09 00:11:12.230000 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.229972 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:12.230218 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.230106 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:12.233624 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.233597 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 00:11:12.234564 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.234015 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 00:11:12.234564 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.234322 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 00:11:12.234564 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.234549 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r4pcw\"" Jun 09 00:11:12.420899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:12.420856 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:12.421059 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:12.421000 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:12.421129 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:12.421061 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:13.42104547 +0000 UTC m=+36.645442716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:13.230384 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:13.230348 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:13.234048 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:13.234020 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 00:11:13.234248 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:13.234024 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ndnwc\"" Jun 09 00:11:13.429963 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:13.429695 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:13.429963 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:13.429850 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:13.429963 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:13.429917 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:15.429897261 +0000 UTC m=+38.654294514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:13.545775 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:13.545745 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gtn7g"] Jun 09 00:11:13.618119 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:11:13.618057 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf29450_7699_4d9e_8505_87f6b8688f9c.slice/crio-918789599cbf6b79134572e75e7728bef7e1fd64a458566735a9d48434909029 WatchSource:0}: Error finding container 918789599cbf6b79134572e75e7728bef7e1fd64a458566735a9d48434909029: Status 404 returned error can't find the container with id 918789599cbf6b79134572e75e7728bef7e1fd64a458566735a9d48434909029 Jun 09 00:11:14.408441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:14.408404 2550 generic.go:356] "Generic (PLEG): container finished" podID="ccd3a06c-34ce-446e-8593-860e6a8eb7c6" containerID="f9a6f8217a1b944276fd446cc12ea6458b4530b3434fdadba65a4ad7fa65373e" exitCode=0 Jun 09 00:11:14.408823 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:14.408488 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerDied","Data":"f9a6f8217a1b944276fd446cc12ea6458b4530b3434fdadba65a4ad7fa65373e"} Jun 09 00:11:14.409737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:14.409715 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gtn7g" event={"ID":"9bf29450-7699-4d9e-8505-87f6b8688f9c","Type":"ContainerStarted","Data":"918789599cbf6b79134572e75e7728bef7e1fd64a458566735a9d48434909029"} Jun 09 00:11:15.416635 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:15.416597 2550 generic.go:356] "Generic (PLEG): container finished" podID="ccd3a06c-34ce-446e-8593-860e6a8eb7c6" containerID="45ad8ee7e6ed8d92e46dbe4aa92633797683fbfa4a79c4e44d0d389e6363f769" exitCode=0 Jun 09 00:11:15.417021 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:15.416636 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerDied","Data":"45ad8ee7e6ed8d92e46dbe4aa92633797683fbfa4a79c4e44d0d389e6363f769"} Jun 09 00:11:15.445213 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:15.445163 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:15.445379 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:15.445311 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:15.445547 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:15.445532 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:19.445358384 +0000 UTC m=+42.669755630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:16.422651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:16.422618 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" event={"ID":"ccd3a06c-34ce-446e-8593-860e6a8eb7c6","Type":"ContainerStarted","Data":"0841130affc3bc8820e7b01ba2dd18263449494a6f3362ad7ce8f40beeef8391"} Jun 09 00:11:16.424146 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:16.424121 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gtn7g" event={"ID":"9bf29450-7699-4d9e-8505-87f6b8688f9c","Type":"ContainerStarted","Data":"436214c6dd429f93637a5349e56d4754a7f7b52fac72cabb2d0ee86a3307f0c4"} Jun 09 00:11:16.448744 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:16.448693 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rt7rv" podStartSLOduration=8.667538299 podStartE2EDuration="39.448678898s" totalImagesPullingTime="30.781140599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:16.446627995 +0000 UTC m=+39.671025264" watchObservedRunningTime="2026-06-09 00:11:16.448678898 +0000 UTC m=+39.673076167" Jun 09 00:11:16.460722 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:16.460674 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gtn7g" podStartSLOduration=3.042990437 podStartE2EDuration="5.460664417s" totalImagesPullingTime="2.41767398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:16.460302671 +0000 UTC m=+39.684699950" watchObservedRunningTime="2026-06-09 00:11:16.460664417 +0000 UTC m=+39.685061684" Jun 09 00:11:19.476609 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:19.476571 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:19.477117 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:19.476684 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:19.477117 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:19.476738 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:27.476725697 +0000 UTC m=+50.701122943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:27.533628 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:27.533591 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:27.534104 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:27.533743 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:27.534104 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:27.533817 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:11:43.53380122 +0000 UTC m=+66.758198470 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:38.402787 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:38.402755 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wks97" Jun 09 00:11:42.942050 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:42.941992 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:42.942505 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:42.942070 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:11:42.944703 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:42.944680 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 00:11:42.944785 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:42.944703 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 00:11:42.952673 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:42.952651 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 00:11:42.952774 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:42.952732 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:12:46.95271035 +0000 UTC m=+130.177107597 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : secret "metrics-daemon-secret" not found Jun 09 00:11:42.955992 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:42.955970 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f62773d3-957a-4951-8974-a26ee5ebd3cb-original-pull-secret\") pod \"global-pull-secret-syncer-x8h82\" (UID: \"f62773d3-957a-4951-8974-a26ee5ebd3cb\") " pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:43.043327 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.043286 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:43.046140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.046123 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 00:11:43.056322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.056298 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 00:11:43.066601 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.066575 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w8jg\" (UniqueName: \"kubernetes.io/projected/9df01a83-e004-4e3f-aa1f-8a928873c214-kube-api-access-8w8jg\") pod \"network-check-target-jbvbz\" (UID: \"9df01a83-e004-4e3f-aa1f-8a928873c214\") " pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:43.146398 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.146367 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r4pcw\"" Jun 09 00:11:43.150102 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.150070 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x8h82" Jun 09 00:11:43.155043 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.155025 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:43.285064 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.285030 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x8h82"] Jun 09 00:11:43.287811 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:11:43.287788 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62773d3_957a_4951_8974_a26ee5ebd3cb.slice/crio-6c1cb55c04619c4b696cd5b18875880c329341b7a7f401313b6e3bd720cea880 WatchSource:0}: Error finding container 6c1cb55c04619c4b696cd5b18875880c329341b7a7f401313b6e3bd720cea880: Status 404 returned error can't find the container with id 6c1cb55c04619c4b696cd5b18875880c329341b7a7f401313b6e3bd720cea880 Jun 09 00:11:43.298718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.298694 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbvbz"] Jun 09 00:11:43.302064 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:11:43.302041 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df01a83_e004_4e3f_aa1f_8a928873c214.slice/crio-72c23c798e69bec2c12b60b5c51c34d74cebb34b1ceabd4f6b0449dce337eb84 WatchSource:0}: Error finding container 72c23c798e69bec2c12b60b5c51c34d74cebb34b1ceabd4f6b0449dce337eb84: Status 404 returned error can't find the container with id 72c23c798e69bec2c12b60b5c51c34d74cebb34b1ceabd4f6b0449dce337eb84 Jun 09 00:11:43.487187 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.487152 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbvbz" event={"ID":"9df01a83-e004-4e3f-aa1f-8a928873c214","Type":"ContainerStarted","Data":"72c23c798e69bec2c12b60b5c51c34d74cebb34b1ceabd4f6b0449dce337eb84"} Jun 09 00:11:43.488227 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.488206 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x8h82" event={"ID":"f62773d3-957a-4951-8974-a26ee5ebd3cb","Type":"ContainerStarted","Data":"6c1cb55c04619c4b696cd5b18875880c329341b7a7f401313b6e3bd720cea880"} Jun 09 00:11:43.546945 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:43.546907 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:11:43.547130 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:43.547109 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:11:43.547205 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:11:43.547190 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:12:15.547168268 +0000 UTC m=+98.771565517 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:11:48.503569 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:48.503523 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbvbz" event={"ID":"9df01a83-e004-4e3f-aa1f-8a928873c214","Type":"ContainerStarted","Data":"f975de88e35ab140d0a0e98ba63ddb999d6762112207f38e3e9ca2b4352ac362"} Jun 09 00:11:48.504104 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:48.503625 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:11:48.504986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:48.504963 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x8h82" event={"ID":"f62773d3-957a-4951-8974-a26ee5ebd3cb","Type":"ContainerStarted","Data":"14e782c13f428e1424d43b19c622022b5979ed65b8b199e7732ad264450bf908"} Jun 09 00:11:48.523869 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:48.523822 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jbvbz" podStartSLOduration=67.177471148 podStartE2EDuration="1m11.523808261s" totalImagesPullingTime="4.346337113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:48.522723052 +0000 UTC m=+71.747120321" watchObservedRunningTime="2026-06-09 00:11:48.523808261 +0000 UTC m=+71.748205529" Jun 09 00:11:48.545294 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:11:48.545244 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x8h82" podStartSLOduration=67.180121265 podStartE2EDuration="1m11.54523071s" totalImagesPullingTime="4.365109445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:11:48.543665694 +0000 UTC m=+71.768062962" watchObservedRunningTime="2026-06-09 00:11:48.54523071 +0000 UTC m=+71.769628015" Jun 09 00:12:15.564632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:15.564591 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:12:15.565238 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:15.564716 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 00:12:15.565238 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:15.564809 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls podName:541a23f9-36b8-4b1f-a7a3-254f348ec588 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:19.564783642 +0000 UTC m=+162.789180898 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls") pod "dns-default-pd9ph" (UID: "541a23f9-36b8-4b1f-a7a3-254f348ec588") : secret "dns-default-metrics-tls" not found Jun 09 00:12:19.509688 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:19.509656 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jbvbz" Jun 09 00:12:46.983402 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:46.983364 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:12:46.983891 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:46.983489 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 00:12:46.983891 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:46.983557 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs podName:c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0 nodeName:}" failed. No retries permitted until 2026-06-09 00:14:48.98354176 +0000 UTC m=+252.207939006 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs") pod "network-metrics-daemon-vx6qc" (UID: "c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0") : secret "metrics-daemon-secret" not found Jun 09 00:12:58.808020 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.807987 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d"] Jun 09 00:12:58.809930 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.809913 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:58.812408 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.812388 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 09 00:12:58.812494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.812426 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6xwvg\"" Jun 09 00:12:58.813355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.813337 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 09 00:12:58.818163 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.818139 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-556c4bd9f5-skfv4"] Jun 09 00:12:58.819737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.819723 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.822172 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.822157 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 09 00:12:58.822241 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.822169 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jxb56\"" Jun 09 00:12:58.822241 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.822226 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 09 00:12:58.822375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.822359 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 09 00:12:58.826930 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.826912 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d"] Jun 09 00:12:58.828251 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.828234 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 09 00:12:58.850306 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.850277 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-556c4bd9f5-skfv4"] Jun 09 00:12:58.912150 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.912116 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84"] Jun 09 00:12:58.914253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.914235 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:58.916872 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.916849 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-c4jr9\"" Jun 09 00:12:58.916986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.916880 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 09 00:12:58.916986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.916949 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 09 00:12:58.917112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.916950 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 09 00:12:58.917112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.917104 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 09 00:12:58.923099 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.923058 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-jk8dr"] Jun 09 00:12:58.924656 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.924642 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj"] Jun 09 00:12:58.924772 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.924754 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:58.926380 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.926361 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:58.930129 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930104 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 09 00:12:58.930404 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930385 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 00:12:58.930488 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930436 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 09 00:12:58.930488 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930393 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 00:12:58.930660 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930639 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qgrfp\"" Jun 09 00:12:58.930761 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930702 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 09 00:12:58.930824 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.930802 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 09 00:12:58.931469 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.931439 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84"] Jun 09 00:12:58.933689 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.933661 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 09 00:12:58.933689 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.933666 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6bk68\"" Jun 09 00:12:58.937589 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.937567 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 09 00:12:58.942722 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.942694 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj"] Jun 09 00:12:58.944293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.944272 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-jk8dr"] Jun 09 00:12:58.969261 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969224 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-image-registry-private-configuration\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969270 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tljz7\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-kube-api-access-tljz7\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969326 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10b97160-eef2-4d08-8c87-8414b5c990a5-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:58.969441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969355 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969382 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-ca-trust-extracted\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969412 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:58.969441 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969440 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-certificates\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969472 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-installation-pull-secrets\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969510 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-trusted-ca\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:58.969760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:58.969529 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-bound-sa-token\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.004428 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.004396 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd"] Jun 09 00:12:59.008912 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.008888 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" Jun 09 00:12:59.010994 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.010971 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-c9bsm\"" Jun 09 00:12:59.016222 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.016204 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd"] Jun 09 00:12:59.070295 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070218 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2556458f-8fa9-489e-86f4-b52a169ef0dc-trusted-ca-bundle\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.070295 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070255 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2556458f-8fa9-489e-86f4-b52a169ef0dc-serving-cert\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.070295 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070278 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-installation-pull-secrets\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.070528 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070324 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:59.070528 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070354 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dspj\" (UniqueName: \"kubernetes.io/projected/3dac3d19-e205-49ff-8b86-3774e61e7f0e-kube-api-access-8dspj\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:59.070528 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070384 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.070528 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070415 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:59.070528 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070438 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb6d2\" (UniqueName: \"kubernetes.io/projected/fd0813dd-8d69-4799-9338-e9c44900f3c4-kube-api-access-xb6d2\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.070528 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070508 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-trusted-ca\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.070533 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.070546 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070555 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd0813dd-8d69-4799-9338-e9c44900f3c4-config\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.070562 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-556c4bd9f5-skfv4: secret "image-registry-tls" not found Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.070608 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert podName:10b97160-eef2-4d08-8c87-8414b5c990a5 nodeName:}" failed. No retries permitted until 2026-06-09 00:12:59.570586744 +0000 UTC m=+142.794983991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-bfx2d" (UID: "10b97160-eef2-4d08-8c87-8414b5c990a5") : secret "networking-console-plugin-cert" not found Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.070640 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls podName:bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4 nodeName:}" failed. No retries permitted until 2026-06-09 00:12:59.570621644 +0000 UTC m=+142.795018905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls") pod "image-registry-556c4bd9f5-skfv4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4") : secret "image-registry-tls" not found Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070685 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2556458f-8fa9-489e-86f4-b52a169ef0dc-tmp\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070745 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-bound-sa-token\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.070788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070787 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgngj\" (UniqueName: \"kubernetes.io/projected/2556458f-8fa9-489e-86f4-b52a169ef0dc-kube-api-access-xgngj\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070819 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-image-registry-private-configuration\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tljz7\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-kube-api-access-tljz7\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070889 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2556458f-8fa9-489e-86f4-b52a169ef0dc-service-ca-bundle\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.070977 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10b97160-eef2-4d08-8c87-8414b5c990a5-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071002 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd0813dd-8d69-4799-9338-e9c44900f3c4-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071029 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-ca-trust-extracted\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071059 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2556458f-8fa9-489e-86f4-b52a169ef0dc-snapshots\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.071253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071233 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-certificates\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071804 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071393 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-ca-trust-extracted\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071804 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071599 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-trusted-ca\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071804 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071660 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-certificates\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.071959 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.071890 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10b97160-eef2-4d08-8c87-8414b5c990a5-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:59.073035 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.073010 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-installation-pull-secrets\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.073293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.073277 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-image-registry-private-configuration\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.079572 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.079551 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-bound-sa-token\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.079679 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.079662 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tljz7\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-kube-api-access-tljz7\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.171979 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.171943 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgngj\" (UniqueName: \"kubernetes.io/projected/2556458f-8fa9-489e-86f4-b52a169ef0dc-kube-api-access-xgngj\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.171979 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.171989 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2556458f-8fa9-489e-86f4-b52a169ef0dc-service-ca-bundle\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172128 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd0813dd-8d69-4799-9338-e9c44900f3c4-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.172278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172184 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2556458f-8fa9-489e-86f4-b52a169ef0dc-snapshots\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172240 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flqpr\" (UniqueName: \"kubernetes.io/projected/f0ed90be-ef3e-4132-ad10-35679895853c-kube-api-access-flqpr\") pod \"network-check-source-dbd9cf844-r5kqd\" (UID: \"f0ed90be-ef3e-4132-ad10-35679895853c\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" Jun 09 00:12:59.172278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172275 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2556458f-8fa9-489e-86f4-b52a169ef0dc-trusted-ca-bundle\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172298 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2556458f-8fa9-489e-86f4-b52a169ef0dc-serving-cert\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172332 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:59.172494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172352 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dspj\" (UniqueName: \"kubernetes.io/projected/3dac3d19-e205-49ff-8b86-3774e61e7f0e-kube-api-access-8dspj\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:59.172494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172400 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb6d2\" (UniqueName: \"kubernetes.io/projected/fd0813dd-8d69-4799-9338-e9c44900f3c4-kube-api-access-xb6d2\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.172494 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172443 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd0813dd-8d69-4799-9338-e9c44900f3c4-config\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.172494 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.172463 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 09 00:12:59.172775 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.172522 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls podName:3dac3d19-e205-49ff-8b86-3774e61e7f0e nodeName:}" failed. No retries permitted until 2026-06-09 00:12:59.672504632 +0000 UTC m=+142.896901880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-8spnj" (UID: "3dac3d19-e205-49ff-8b86-3774e61e7f0e") : secret "samples-operator-tls" not found Jun 09 00:12:59.172775 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172465 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2556458f-8fa9-489e-86f4-b52a169ef0dc-tmp\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172775 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172721 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2556458f-8fa9-489e-86f4-b52a169ef0dc-service-ca-bundle\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172929 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172898 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2556458f-8fa9-489e-86f4-b52a169ef0dc-snapshots\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.172978 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.172929 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2556458f-8fa9-489e-86f4-b52a169ef0dc-tmp\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.173274 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.173252 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2556458f-8fa9-489e-86f4-b52a169ef0dc-trusted-ca-bundle\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.173328 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.173280 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd0813dd-8d69-4799-9338-e9c44900f3c4-config\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.175257 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.175234 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd0813dd-8d69-4799-9338-e9c44900f3c4-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.175344 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.175303 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2556458f-8fa9-489e-86f4-b52a169ef0dc-serving-cert\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.180543 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.180517 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb6d2\" (UniqueName: \"kubernetes.io/projected/fd0813dd-8d69-4799-9338-e9c44900f3c4-kube-api-access-xb6d2\") pod \"service-ca-operator-6bc5ff68f5-phs84\" (UID: \"fd0813dd-8d69-4799-9338-e9c44900f3c4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.180666 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.180647 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgngj\" (UniqueName: \"kubernetes.io/projected/2556458f-8fa9-489e-86f4-b52a169ef0dc-kube-api-access-xgngj\") pod \"insights-operator-896549976-jk8dr\" (UID: \"2556458f-8fa9-489e-86f4-b52a169ef0dc\") " pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.180909 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.180892 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dspj\" (UniqueName: \"kubernetes.io/projected/3dac3d19-e205-49ff-8b86-3774e61e7f0e-kube-api-access-8dspj\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:59.223728 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.223696 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" Jun 09 00:12:59.235446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.235424 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-jk8dr" Jun 09 00:12:59.274285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.273652 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqpr\" (UniqueName: \"kubernetes.io/projected/f0ed90be-ef3e-4132-ad10-35679895853c-kube-api-access-flqpr\") pod \"network-check-source-dbd9cf844-r5kqd\" (UID: \"f0ed90be-ef3e-4132-ad10-35679895853c\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" Jun 09 00:12:59.282555 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.282526 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flqpr\" (UniqueName: \"kubernetes.io/projected/f0ed90be-ef3e-4132-ad10-35679895853c-kube-api-access-flqpr\") pod \"network-check-source-dbd9cf844-r5kqd\" (UID: \"f0ed90be-ef3e-4132-ad10-35679895853c\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" Jun 09 00:12:59.318437 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.318405 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" Jun 09 00:12:59.360691 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.360658 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84"] Jun 09 00:12:59.365354 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:12:59.365317 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0813dd_8d69_4799_9338_e9c44900f3c4.slice/crio-59e4a37dac4000b7d80f8dc85cd500ce0123752442f2061a6dd0b3f34dc8603d WatchSource:0}: Error finding container 59e4a37dac4000b7d80f8dc85cd500ce0123752442f2061a6dd0b3f34dc8603d: Status 404 returned error can't find the container with id 59e4a37dac4000b7d80f8dc85cd500ce0123752442f2061a6dd0b3f34dc8603d Jun 09 00:12:59.381506 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.381476 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-jk8dr"] Jun 09 00:12:59.384888 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:12:59.384853 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2556458f_8fa9_489e_86f4_b52a169ef0dc.slice/crio-420ef84192386158c4feee95ac30bdad18087d917c77fbe543664b895f0f1a07 WatchSource:0}: Error finding container 420ef84192386158c4feee95ac30bdad18087d917c77fbe543664b895f0f1a07: Status 404 returned error can't find the container with id 420ef84192386158c4feee95ac30bdad18087d917c77fbe543664b895f0f1a07 Jun 09 00:12:59.453669 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.453631 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd"] Jun 09 00:12:59.457173 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:12:59.457135 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ed90be_ef3e_4132_ad10_35679895853c.slice/crio-88dd12756c1c35079c1c13d0e06333d032f43d01863d2ed6e8d597e61a0cf0df WatchSource:0}: Error finding container 88dd12756c1c35079c1c13d0e06333d032f43d01863d2ed6e8d597e61a0cf0df: Status 404 returned error can't find the container with id 88dd12756c1c35079c1c13d0e06333d032f43d01863d2ed6e8d597e61a0cf0df Jun 09 00:12:59.575793 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.575702 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:12:59.575793 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.575746 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:12:59.576004 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.575840 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 00:12:59.576004 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.575857 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-556c4bd9f5-skfv4: secret "image-registry-tls" not found Jun 09 00:12:59.576004 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.575883 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 00:12:59.576004 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.575909 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls podName:bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:00.575895533 +0000 UTC m=+143.800292780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls") pod "image-registry-556c4bd9f5-skfv4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4") : secret "image-registry-tls" not found Jun 09 00:12:59.576004 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.575935 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert podName:10b97160-eef2-4d08-8c87-8414b5c990a5 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:00.575921132 +0000 UTC m=+143.800318379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-bfx2d" (UID: "10b97160-eef2-4d08-8c87-8414b5c990a5") : secret "networking-console-plugin-cert" not found Jun 09 00:12:59.668964 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.668920 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" event={"ID":"f0ed90be-ef3e-4132-ad10-35679895853c","Type":"ContainerStarted","Data":"7c9030e32dd36f4fa8d3be84471bfe8745caab5576a26c73e9aa6a5f01fc4b0f"} Jun 09 00:12:59.669208 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.668982 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" event={"ID":"f0ed90be-ef3e-4132-ad10-35679895853c","Type":"ContainerStarted","Data":"88dd12756c1c35079c1c13d0e06333d032f43d01863d2ed6e8d597e61a0cf0df"} Jun 09 00:12:59.670138 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.670110 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" event={"ID":"fd0813dd-8d69-4799-9338-e9c44900f3c4","Type":"ContainerStarted","Data":"59e4a37dac4000b7d80f8dc85cd500ce0123752442f2061a6dd0b3f34dc8603d"} Jun 09 00:12:59.671034 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.671013 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jk8dr" event={"ID":"2556458f-8fa9-489e-86f4-b52a169ef0dc","Type":"ContainerStarted","Data":"420ef84192386158c4feee95ac30bdad18087d917c77fbe543664b895f0f1a07"} Jun 09 00:12:59.676242 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.676223 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:12:59.676335 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.676323 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 09 00:12:59.676380 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:12:59.676373 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls podName:3dac3d19-e205-49ff-8b86-3774e61e7f0e nodeName:}" failed. No retries permitted until 2026-06-09 00:13:00.67636048 +0000 UTC m=+143.900757726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-8spnj" (UID: "3dac3d19-e205-49ff-8b86-3774e61e7f0e") : secret "samples-operator-tls" not found Jun 09 00:12:59.683353 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:12:59.683312 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-r5kqd" podStartSLOduration=1.683298777 podStartE2EDuration="1.683298777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:12:59.682537435 +0000 UTC m=+142.906934705" watchObservedRunningTime="2026-06-09 00:12:59.683298777 +0000 UTC m=+142.907696054" Jun 09 00:13:00.583380 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:00.583182 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:00.583380 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:00.583234 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:13:00.583380 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.583332 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 00:13:00.583380 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.583357 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-556c4bd9f5-skfv4: secret "image-registry-tls" not found Jun 09 00:13:00.583380 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.583377 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 00:13:00.584111 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.583430 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls podName:bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:02.583407801 +0000 UTC m=+145.807805061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls") pod "image-registry-556c4bd9f5-skfv4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4") : secret "image-registry-tls" not found Jun 09 00:13:00.584111 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.583450 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert podName:10b97160-eef2-4d08-8c87-8414b5c990a5 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:02.583440274 +0000 UTC m=+145.807837533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-bfx2d" (UID: "10b97160-eef2-4d08-8c87-8414b5c990a5") : secret "networking-console-plugin-cert" not found Jun 09 00:13:00.684291 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:00.684254 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:13:00.684481 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.684414 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 09 00:13:00.684546 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:00.684484 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls podName:3dac3d19-e205-49ff-8b86-3774e61e7f0e nodeName:}" failed. No retries permitted until 2026-06-09 00:13:02.684463319 +0000 UTC m=+145.908860580 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-8spnj" (UID: "3dac3d19-e205-49ff-8b86-3774e61e7f0e") : secret "samples-operator-tls" not found Jun 09 00:13:02.599133 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.599070 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:02.599133 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.599138 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:13:02.599664 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.599216 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 00:13:02.599664 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.599235 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-556c4bd9f5-skfv4: secret "image-registry-tls" not found Jun 09 00:13:02.599664 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.599285 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 00:13:02.599664 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.599289 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls podName:bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:06.599275389 +0000 UTC m=+149.823672638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls") pod "image-registry-556c4bd9f5-skfv4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4") : secret "image-registry-tls" not found Jun 09 00:13:02.599664 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.599347 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert podName:10b97160-eef2-4d08-8c87-8414b5c990a5 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:06.599333423 +0000 UTC m=+149.823730672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-bfx2d" (UID: "10b97160-eef2-4d08-8c87-8414b5c990a5") : secret "networking-console-plugin-cert" not found Jun 09 00:13:02.680544 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.680459 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jk8dr" event={"ID":"2556458f-8fa9-489e-86f4-b52a169ef0dc","Type":"ContainerStarted","Data":"7fa0a323378d8f148312bad40c40b9b85beb65b0b3ee56cddd06886078146429"} Jun 09 00:13:02.686894 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.686856 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" event={"ID":"fd0813dd-8d69-4799-9338-e9c44900f3c4","Type":"ContainerStarted","Data":"121a441ba960d43860d74e5b84d0b673bdcc6db8379d45027f0bd1b22bfc6008"} Jun 09 00:13:02.696355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.696298 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-jk8dr" podStartSLOduration=1.7143549519999999 podStartE2EDuration="4.696280354s" totalImagesPullingTime="2.981925402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:02.694726516 +0000 UTC m=+145.919123778" watchObservedRunningTime="2026-06-09 00:13:02.696280354 +0000 UTC m=+145.920677624" Jun 09 00:13:02.700421 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.700398 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:13:02.700553 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.700535 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 09 00:13:02.700609 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:02.700600 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls podName:3dac3d19-e205-49ff-8b86-3774e61e7f0e nodeName:}" failed. No retries permitted until 2026-06-09 00:13:06.700583073 +0000 UTC m=+149.924980334 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-8spnj" (UID: "3dac3d19-e205-49ff-8b86-3774e61e7f0e") : secret "samples-operator-tls" not found Jun 09 00:13:02.708095 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:02.708030 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" podStartSLOduration=1.705500156 podStartE2EDuration="4.708012627s" totalImagesPullingTime="3.002512471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:02.707491646 +0000 UTC m=+145.931888916" watchObservedRunningTime="2026-06-09 00:13:02.708012627 +0000 UTC m=+145.932409896" Jun 09 00:13:05.862720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.862679 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-thjmk"] Jun 09 00:13:05.865702 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.865679 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:05.868450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.868427 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 09 00:13:05.869166 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.869146 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 09 00:13:05.869166 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.869161 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 09 00:13:05.869257 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.869171 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r2hjr\"" Jun 09 00:13:05.869257 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.869196 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 09 00:13:05.897043 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:05.897010 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-thjmk"] Jun 09 00:13:06.027843 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.027807 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72n7m\" (UniqueName: \"kubernetes.io/projected/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-kube-api-access-72n7m\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.028012 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.027935 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-signing-cabundle\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.028012 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.027993 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-signing-key\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.128790 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.128668 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72n7m\" (UniqueName: \"kubernetes.io/projected/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-kube-api-access-72n7m\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.128790 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.128782 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-signing-cabundle\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.129029 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.128837 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-signing-key\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.129457 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.129435 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-signing-cabundle\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.131504 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.131482 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-signing-key\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.138256 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.138232 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72n7m\" (UniqueName: \"kubernetes.io/projected/205dfffa-db5b-4128-a6a4-bbfe900cc2c0-kube-api-access-72n7m\") pod \"service-ca-7d74d578b8-thjmk\" (UID: \"205dfffa-db5b-4128-a6a4-bbfe900cc2c0\") " pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.175072 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.175034 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" Jun 09 00:13:06.303876 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.303839 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-thjmk"] Jun 09 00:13:06.308578 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:06.308548 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205dfffa_db5b_4128_a6a4_bbfe900cc2c0.slice/crio-683dcc5d60fe76f99ea95b6c5434b26f6a248b92a728d013c83332856e77f350 WatchSource:0}: Error finding container 683dcc5d60fe76f99ea95b6c5434b26f6a248b92a728d013c83332856e77f350: Status 404 returned error can't find the container with id 683dcc5d60fe76f99ea95b6c5434b26f6a248b92a728d013c83332856e77f350 Jun 09 00:13:06.602471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.602443 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g97wc_dc81db3e-2bb4-42bc-9b8d-c07d09d3226e/dns-node-resolver/0.log" Jun 09 00:13:06.633380 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.633348 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:06.633551 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.633385 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:13:06.633551 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.633508 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 00:13:06.633551 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.633517 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 00:13:06.633551 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.633527 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-556c4bd9f5-skfv4: secret "image-registry-tls" not found Jun 09 00:13:06.633708 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.633574 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert podName:10b97160-eef2-4d08-8c87-8414b5c990a5 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:14.633561458 +0000 UTC m=+157.857958704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-bfx2d" (UID: "10b97160-eef2-4d08-8c87-8414b5c990a5") : secret "networking-console-plugin-cert" not found Jun 09 00:13:06.633708 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.633587 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls podName:bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:14.633580903 +0000 UTC m=+157.857978149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls") pod "image-registry-556c4bd9f5-skfv4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4") : secret "image-registry-tls" not found Jun 09 00:13:06.701155 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.701117 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" event={"ID":"205dfffa-db5b-4128-a6a4-bbfe900cc2c0","Type":"ContainerStarted","Data":"4d3f4c8134c651ca63333aed5b26fb2be5feef9111ef061974267945bce695ec"} Jun 09 00:13:06.701348 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.701174 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" event={"ID":"205dfffa-db5b-4128-a6a4-bbfe900cc2c0","Type":"ContainerStarted","Data":"683dcc5d60fe76f99ea95b6c5434b26f6a248b92a728d013c83332856e77f350"} Jun 09 00:13:06.725398 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.725341 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-thjmk" podStartSLOduration=1.72532685 podStartE2EDuration="1.72532685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:06.72453159 +0000 UTC m=+149.948928858" watchObservedRunningTime="2026-06-09 00:13:06.72532685 +0000 UTC m=+149.949724118" Jun 09 00:13:06.734563 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:06.734520 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:13:06.734737 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.734713 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 09 00:13:06.734805 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:06.734795 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls podName:3dac3d19-e205-49ff-8b86-3774e61e7f0e nodeName:}" failed. No retries permitted until 2026-06-09 00:13:14.734776048 +0000 UTC m=+157.959173309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-8spnj" (UID: "3dac3d19-e205-49ff-8b86-3774e61e7f0e") : secret "samples-operator-tls" not found Jun 09 00:13:07.603509 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:07.603476 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nj5z7_3e0a6921-f28b-42f1-a2c9-05ed1e65c4de/node-ca/0.log" Jun 09 00:13:08.402768 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:08.402743 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gtn7g_9bf29450-7699-4d9e-8505-87f6b8688f9c/serve-healthcheck-canary/0.log" Jun 09 00:13:14.703507 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.703404 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:14.703507 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.703457 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:13:14.705925 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.705897 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"image-registry-556c4bd9f5-skfv4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:14.706028 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.705999 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10b97160-eef2-4d08-8c87-8414b5c990a5-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-bfx2d\" (UID: \"10b97160-eef2-4d08-8c87-8414b5c990a5\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:13:14.706923 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:14.706895 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pd9ph" podUID="541a23f9-36b8-4b1f-a7a3-254f348ec588" Jun 09 00:13:14.718929 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.718909 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" Jun 09 00:13:14.727331 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.727307 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pd9ph" Jun 09 00:13:14.728727 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.728710 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:14.804336 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.804242 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:13:14.806733 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.806711 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dac3d19-e205-49ff-8b86-3774e61e7f0e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-8spnj\" (UID: \"3dac3d19-e205-49ff-8b86-3774e61e7f0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:13:14.840749 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.840713 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" Jun 09 00:13:14.858604 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.858549 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d"] Jun 09 00:13:14.864030 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:14.864005 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b97160_eef2_4d08_8c87_8414b5c990a5.slice/crio-ee738f541081d2b01bd2a7dc3da2ce9aeb923f7024ab09fd370a36129349a80d WatchSource:0}: Error finding container ee738f541081d2b01bd2a7dc3da2ce9aeb923f7024ab09fd370a36129349a80d: Status 404 returned error can't find the container with id ee738f541081d2b01bd2a7dc3da2ce9aeb923f7024ab09fd370a36129349a80d Jun 09 00:13:14.878404 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.878379 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-556c4bd9f5-skfv4"] Jun 09 00:13:14.881603 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:14.881572 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd06ca0f_5c26_47d7_ae03_1a1980e0f7b4.slice/crio-64f6abf47c95df101753c79b45f2f9be7e9a8a651675f6145cef05d1804d5cd7 WatchSource:0}: Error finding container 64f6abf47c95df101753c79b45f2f9be7e9a8a651675f6145cef05d1804d5cd7: Status 404 returned error can't find the container with id 64f6abf47c95df101753c79b45f2f9be7e9a8a651675f6145cef05d1804d5cd7 Jun 09 00:13:14.979940 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:14.979894 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj"] Jun 09 00:13:15.732493 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:15.732454 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" event={"ID":"3dac3d19-e205-49ff-8b86-3774e61e7f0e","Type":"ContainerStarted","Data":"c11b61f1cafca16d7c5d09ae2051e3272aaad439ff3860d3a1deabcf37f06353"} Jun 09 00:13:15.734170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:15.734142 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" event={"ID":"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4","Type":"ContainerStarted","Data":"c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417"} Jun 09 00:13:15.734324 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:15.734190 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" event={"ID":"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4","Type":"ContainerStarted","Data":"64f6abf47c95df101753c79b45f2f9be7e9a8a651675f6145cef05d1804d5cd7"} Jun 09 00:13:15.734384 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:15.734334 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:15.735769 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:15.735740 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" event={"ID":"10b97160-eef2-4d08-8c87-8414b5c990a5","Type":"ContainerStarted","Data":"ee738f541081d2b01bd2a7dc3da2ce9aeb923f7024ab09fd370a36129349a80d"} Jun 09 00:13:15.753998 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:15.753662 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" podStartSLOduration=17.753643367 podStartE2EDuration="17.753643367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:15.752063788 +0000 UTC m=+158.976461058" watchObservedRunningTime="2026-06-09 00:13:15.753643367 +0000 UTC m=+158.978040637" Jun 09 00:13:16.240874 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:16.240829 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vx6qc" podUID="c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0" Jun 09 00:13:16.741643 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:16.741579 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" event={"ID":"10b97160-eef2-4d08-8c87-8414b5c990a5","Type":"ContainerStarted","Data":"a5aa1dd89421343b6c96500cb4508efd602ec3cfd6f587690c174e2c7d4e8498"} Jun 09 00:13:16.756880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:16.756823 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-bfx2d" podStartSLOduration=17.331569854 podStartE2EDuration="18.756802235s" totalImagesPullingTime="1.425232381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:16.756169418 +0000 UTC m=+159.980566688" watchObservedRunningTime="2026-06-09 00:13:16.756802235 +0000 UTC m=+159.981199515" Jun 09 00:13:17.746007 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:17.745968 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" event={"ID":"3dac3d19-e205-49ff-8b86-3774e61e7f0e","Type":"ContainerStarted","Data":"f918ea87c2e260ae330d247c125d04caff93d68c36801130257e148f9909123f"} Jun 09 00:13:17.746410 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:17.746020 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" event={"ID":"3dac3d19-e205-49ff-8b86-3774e61e7f0e","Type":"ContainerStarted","Data":"6c0855318eb7082c76c543a3dcd4b4988ac5b6991b737d4ffb07ad80eb68e92e"} Jun 09 00:13:17.761202 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:17.761161 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-8spnj" podStartSLOduration=17.617201363 podStartE2EDuration="19.761146701s" totalImagesPullingTime="2.143945338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:17.759930744 +0000 UTC m=+160.984328012" watchObservedRunningTime="2026-06-09 00:13:17.761146701 +0000 UTC m=+160.985543968" Jun 09 00:13:19.648275 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:19.648232 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:13:19.650633 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:19.650605 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/541a23f9-36b8-4b1f-a7a3-254f348ec588-metrics-tls\") pod \"dns-default-pd9ph\" (UID: \"541a23f9-36b8-4b1f-a7a3-254f348ec588\") " pod="openshift-dns/dns-default-pd9ph" Jun 09 00:13:19.830343 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:19.830309 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hv7fp\"" Jun 09 00:13:19.838344 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:19.838313 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pd9ph" Jun 09 00:13:19.966408 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:19.966290 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pd9ph"] Jun 09 00:13:19.968733 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:19.968706 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541a23f9_36b8_4b1f_a7a3_254f348ec588.slice/crio-dad9d0be3d9db27e1b2779223830daadb5da50c736d83fcb411cb977af351ded WatchSource:0}: Error finding container dad9d0be3d9db27e1b2779223830daadb5da50c736d83fcb411cb977af351ded: Status 404 returned error can't find the container with id dad9d0be3d9db27e1b2779223830daadb5da50c736d83fcb411cb977af351ded Jun 09 00:13:20.757127 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:20.757092 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pd9ph" event={"ID":"541a23f9-36b8-4b1f-a7a3-254f348ec588","Type":"ContainerStarted","Data":"dad9d0be3d9db27e1b2779223830daadb5da50c736d83fcb411cb977af351ded"} Jun 09 00:13:22.768935 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:22.768893 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pd9ph" event={"ID":"541a23f9-36b8-4b1f-a7a3-254f348ec588","Type":"ContainerStarted","Data":"210ed8a47dec00979011c0a3e275642f8b1ce8bdf0a941994e5d3b20e517a6d3"} Jun 09 00:13:22.769355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:22.768973 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pd9ph" event={"ID":"541a23f9-36b8-4b1f-a7a3-254f348ec588","Type":"ContainerStarted","Data":"87a8765388846abd73623f54ccc03bb2d81ceaef7a68b1a1a069a6bf43a03bf6"} Jun 09 00:13:22.769355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:22.769054 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pd9ph" Jun 09 00:13:22.787426 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:22.787372 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pd9ph" podStartSLOduration=129.876709607 podStartE2EDuration="2m11.787355987s" totalImagesPullingTime="1.91064638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:22.78557194 +0000 UTC m=+166.009969223" watchObservedRunningTime="2026-06-09 00:13:22.787355987 +0000 UTC m=+166.011753254" Jun 09 00:13:27.231002 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:27.230956 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:13:28.589165 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.589132 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-556c4bd9f5-skfv4"] Jun 09 00:13:28.591806 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.591778 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n5ktt"] Jun 09 00:13:28.595099 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.595063 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.599254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.599228 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 00:13:28.600321 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.600293 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 00:13:28.601993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.601976 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2r7fp\"" Jun 09 00:13:28.609279 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.609253 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5ktt"] Jun 09 00:13:28.722411 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.722380 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e767f1c2-5f25-4bd2-bcf4-571000193825-crio-socket\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.722626 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.722435 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8ghg\" (UniqueName: \"kubernetes.io/projected/e767f1c2-5f25-4bd2-bcf4-571000193825-kube-api-access-g8ghg\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.722626 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.722502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e767f1c2-5f25-4bd2-bcf4-571000193825-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.722626 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.722568 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e767f1c2-5f25-4bd2-bcf4-571000193825-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.722626 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.722605 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e767f1c2-5f25-4bd2-bcf4-571000193825-data-volume\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.823571 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.823537 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e767f1c2-5f25-4bd2-bcf4-571000193825-data-volume\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.823571 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.823572 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e767f1c2-5f25-4bd2-bcf4-571000193825-crio-socket\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.823814 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.823603 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8ghg\" (UniqueName: \"kubernetes.io/projected/e767f1c2-5f25-4bd2-bcf4-571000193825-kube-api-access-g8ghg\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.823814 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.823645 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e767f1c2-5f25-4bd2-bcf4-571000193825-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.823814 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.823671 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e767f1c2-5f25-4bd2-bcf4-571000193825-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.823814 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.823685 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e767f1c2-5f25-4bd2-bcf4-571000193825-crio-socket\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.824021 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.824007 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e767f1c2-5f25-4bd2-bcf4-571000193825-data-volume\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.824213 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.824191 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e767f1c2-5f25-4bd2-bcf4-571000193825-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.826244 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.826222 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e767f1c2-5f25-4bd2-bcf4-571000193825-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.835220 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.835200 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8ghg\" (UniqueName: \"kubernetes.io/projected/e767f1c2-5f25-4bd2-bcf4-571000193825-kube-api-access-g8ghg\") pod \"insights-runtime-extractor-n5ktt\" (UID: \"e767f1c2-5f25-4bd2-bcf4-571000193825\") " pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:28.904937 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:28.904851 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5ktt" Jun 09 00:13:29.036030 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:29.035985 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5ktt"] Jun 09 00:13:29.061950 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:29.061913 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode767f1c2_5f25_4bd2_bcf4_571000193825.slice/crio-34c8b89dd126933f494a8fc409db51a571ce46278dad6a971c57398b2d1152d8 WatchSource:0}: Error finding container 34c8b89dd126933f494a8fc409db51a571ce46278dad6a971c57398b2d1152d8: Status 404 returned error can't find the container with id 34c8b89dd126933f494a8fc409db51a571ce46278dad6a971c57398b2d1152d8 Jun 09 00:13:29.794146 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:29.794106 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5ktt" event={"ID":"e767f1c2-5f25-4bd2-bcf4-571000193825","Type":"ContainerStarted","Data":"5db8d5f103a233878c7b2438e19c9721134c8207c2ca5480556510d2e3a3eb19"} Jun 09 00:13:29.794537 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:29.794168 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5ktt" event={"ID":"e767f1c2-5f25-4bd2-bcf4-571000193825","Type":"ContainerStarted","Data":"34c8b89dd126933f494a8fc409db51a571ce46278dad6a971c57398b2d1152d8"} Jun 09 00:13:30.800533 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.800485 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5ktt" event={"ID":"e767f1c2-5f25-4bd2-bcf4-571000193825","Type":"ContainerStarted","Data":"d1aa52732c6132b0cea635533404a6bbd2317f05c94ad12096c230952ce037d0"} Jun 09 00:13:30.960013 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.959982 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-rc55j"] Jun 09 00:13:30.963197 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.963175 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:30.967196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.966874 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 09 00:13:30.967196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.966890 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 09 00:13:30.967196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.966976 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 00:13:30.967196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.966990 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 00:13:30.967196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.966891 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jvrgl\"" Jun 09 00:13:30.967196 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.967106 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 00:13:30.978374 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:30.974453 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-rc55j"] Jun 09 00:13:31.043690 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.043649 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.043865 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.043709 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88d98b0f-1cf2-4794-8301-e3661a6048be-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.043865 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.043736 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.043865 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.043756 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fhn7\" (UniqueName: \"kubernetes.io/projected/88d98b0f-1cf2-4794-8301-e3661a6048be-kube-api-access-6fhn7\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.144582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.144481 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.144582 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.144546 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88d98b0f-1cf2-4794-8301-e3661a6048be-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.144813 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.144596 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.144813 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.144625 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fhn7\" (UniqueName: \"kubernetes.io/projected/88d98b0f-1cf2-4794-8301-e3661a6048be-kube-api-access-6fhn7\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.144813 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:31.144657 2550 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 09 00:13:31.144813 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:31.144733 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-tls podName:88d98b0f-1cf2-4794-8301-e3661a6048be nodeName:}" failed. No retries permitted until 2026-06-09 00:13:31.644717101 +0000 UTC m=+174.869114346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-rc55j" (UID: "88d98b0f-1cf2-4794-8301-e3661a6048be") : secret "prometheus-operator-tls" not found Jun 09 00:13:31.145378 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.145349 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88d98b0f-1cf2-4794-8301-e3661a6048be-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.147783 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.147756 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.152634 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.152594 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fhn7\" (UniqueName: \"kubernetes.io/projected/88d98b0f-1cf2-4794-8301-e3661a6048be-kube-api-access-6fhn7\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.648430 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.648399 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.650803 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.650783 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/88d98b0f-1cf2-4794-8301-e3661a6048be-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-rc55j\" (UID: \"88d98b0f-1cf2-4794-8301-e3661a6048be\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:31.805285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.805197 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5ktt" event={"ID":"e767f1c2-5f25-4bd2-bcf4-571000193825","Type":"ContainerStarted","Data":"73a6e701e97997a759fdcf254230995dd6e05ce62beed852767576aa4f3978da"} Jun 09 00:13:31.821885 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.821838 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n5ktt" podStartSLOduration=1.491585286 podStartE2EDuration="3.821822461s" totalImagesPullingTime="2.330237175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:13:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:31.820954718 +0000 UTC m=+175.045352012" watchObservedRunningTime="2026-06-09 00:13:31.821822461 +0000 UTC m=+175.046219728" Jun 09 00:13:31.876651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:31.876614 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" Jun 09 00:13:32.004506 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:32.004461 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-rc55j"] Jun 09 00:13:32.007020 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:32.006993 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d98b0f_1cf2_4794_8301_e3661a6048be.slice/crio-1f117847e4c0629501e642c926ad3f80c18b557030fa9f98f64a08c5fe6ebd4f WatchSource:0}: Error finding container 1f117847e4c0629501e642c926ad3f80c18b557030fa9f98f64a08c5fe6ebd4f: Status 404 returned error can't find the container with id 1f117847e4c0629501e642c926ad3f80c18b557030fa9f98f64a08c5fe6ebd4f Jun 09 00:13:32.774720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:32.774688 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pd9ph" Jun 09 00:13:32.810680 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:32.810540 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" event={"ID":"88d98b0f-1cf2-4794-8301-e3661a6048be","Type":"ContainerStarted","Data":"1f117847e4c0629501e642c926ad3f80c18b557030fa9f98f64a08c5fe6ebd4f"} Jun 09 00:13:33.816107 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:33.816048 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" event={"ID":"88d98b0f-1cf2-4794-8301-e3661a6048be","Type":"ContainerStarted","Data":"ee9553fa095e162e14e2b98c885d30e9226c59f0b9402d87475a873f1a34b24f"} Jun 09 00:13:33.816536 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:33.816132 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" event={"ID":"88d98b0f-1cf2-4794-8301-e3661a6048be","Type":"ContainerStarted","Data":"e68d188dcdbd3ab471c1665399e6e9d890c18f123c3fb802813e95ecb7dcf4cc"} Jun 09 00:13:33.831522 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:33.831475 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-rc55j" podStartSLOduration=2.41315367 podStartE2EDuration="3.83145958s" totalImagesPullingTime="1.41830591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:33.830425693 +0000 UTC m=+177.054822960" watchObservedRunningTime="2026-06-09 00:13:33.83145958 +0000 UTC m=+177.055856849" Jun 09 00:13:34.329617 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.329587 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-679bf7467b-jj7dt"] Jun 09 00:13:34.332711 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.332694 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.335029 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335001 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 09 00:13:34.335180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335035 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dhtkt\"" Jun 09 00:13:34.335885 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335865 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 09 00:13:34.335981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335911 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 09 00:13:34.335981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335926 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 09 00:13:34.335981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335933 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 09 00:13:34.335981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.335924 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 09 00:13:34.336282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.336268 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 09 00:13:34.341093 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.341056 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679bf7467b-jj7dt"] Jun 09 00:13:34.472767 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.472730 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-service-ca\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.472944 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.472786 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjrgt\" (UniqueName: \"kubernetes.io/projected/fced078e-da4b-404c-aec6-05b16bef129e-kube-api-access-kjrgt\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.472944 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.472850 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-oauth-serving-cert\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.472944 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.472893 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-console-config\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.472944 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.472927 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-serving-cert\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.473123 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.472946 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-oauth-config\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574197 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574162 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-serving-cert\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574197 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-oauth-config\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574425 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574241 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-service-ca\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574425 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574385 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjrgt\" (UniqueName: \"kubernetes.io/projected/fced078e-da4b-404c-aec6-05b16bef129e-kube-api-access-kjrgt\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574524 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574436 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-oauth-serving-cert\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574524 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574488 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-console-config\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.574992 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.574971 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-service-ca\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.575101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.575016 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-console-config\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.575377 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.575356 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-oauth-serving-cert\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.576613 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.576592 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-oauth-config\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.576863 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.576843 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-serving-cert\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.581889 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.581834 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjrgt\" (UniqueName: \"kubernetes.io/projected/fced078e-da4b-404c-aec6-05b16bef129e-kube-api-access-kjrgt\") pod \"console-679bf7467b-jj7dt\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.641985 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.641949 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:34.771492 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.771462 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679bf7467b-jj7dt"] Jun 09 00:13:34.774425 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:34.774388 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfced078e_da4b_404c_aec6_05b16bef129e.slice/crio-83dd697211c62adac08f1fff038d1ae0af246f62ec0e375588034e08d8b04511 WatchSource:0}: Error finding container 83dd697211c62adac08f1fff038d1ae0af246f62ec0e375588034e08d8b04511: Status 404 returned error can't find the container with id 83dd697211c62adac08f1fff038d1ae0af246f62ec0e375588034e08d8b04511 Jun 09 00:13:34.820796 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:34.820758 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679bf7467b-jj7dt" event={"ID":"fced078e-da4b-404c-aec6-05b16bef129e","Type":"ContainerStarted","Data":"83dd697211c62adac08f1fff038d1ae0af246f62ec0e375588034e08d8b04511"} Jun 09 00:13:35.309948 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.309908 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz"] Jun 09 00:13:35.315044 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.315016 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.317538 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.317500 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 09 00:13:35.317753 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.317734 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hswjd\"" Jun 09 00:13:35.318450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.318414 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 09 00:13:35.318881 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.318854 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 09 00:13:35.321471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.321447 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4xx8m"] Jun 09 00:13:35.325431 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.325351 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.326421 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.326399 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz"] Jun 09 00:13:35.328029 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.328005 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qr8k9\"" Jun 09 00:13:35.328247 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.328231 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 00:13:35.328400 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.328382 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 00:13:35.328578 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.328563 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 00:13:35.481920 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.481887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5575c1d-f464-4425-aced-8212ec8efe84-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.481927 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.481987 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-metrics-client-ca\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482018 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-wtmp\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482038 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482060 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2mt7\" (UniqueName: \"kubernetes.io/projected/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-kube-api-access-w2mt7\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482199 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-sys\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482262 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5575c1d-f464-4425-aced-8212ec8efe84-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482293 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482320 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-accelerators-collector-config\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482398 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsk7q\" (UniqueName: \"kubernetes.io/projected/b5575c1d-f464-4425-aced-8212ec8efe84-kube-api-access-lsk7q\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.482446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-tls\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.483027 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482495 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.483027 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482522 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-textfile\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.483027 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.482548 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-root\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583714 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583625 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-root\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583714 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583668 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5575c1d-f464-4425-aced-8212ec8efe84-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.583714 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583686 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583717 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-metrics-client-ca\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583748 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-wtmp\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583763 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-root\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583875 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583921 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2mt7\" (UniqueName: \"kubernetes.io/projected/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-kube-api-access-w2mt7\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583883 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-wtmp\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.583997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.583996 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-sys\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584047 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5575c1d-f464-4425-aced-8212ec8efe84-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584074 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584118 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-accelerators-collector-config\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584144 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5575c1d-f464-4425-aced-8212ec8efe84-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584162 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsk7q\" (UniqueName: \"kubernetes.io/projected/b5575c1d-f464-4425-aced-8212ec8efe84-kube-api-access-lsk7q\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584202 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-tls\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584274 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584302 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-textfile\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584414 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-metrics-client-ca\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584920 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584578 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-textfile\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584920 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584601 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-sys\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.584920 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:35.584752 2550 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 09 00:13:35.584920 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:35.584812 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-tls podName:b5575c1d-f464-4425-aced-8212ec8efe84 nodeName:}" failed. No retries permitted until 2026-06-09 00:13:36.084793967 +0000 UTC m=+179.309191219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-tls") pod "kube-state-metrics-57bbf8bfb5-txxtz" (UID: "b5575c1d-f464-4425-aced-8212ec8efe84") : secret "kube-state-metrics-tls" not found Jun 09 00:13:35.585170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.584947 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5575c1d-f464-4425-aced-8212ec8efe84-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.585251 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.585222 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.585619 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.585590 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-accelerators-collector-config\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.586744 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.586715 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.587112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.587065 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.587455 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.587434 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-node-exporter-tls\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.591792 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.591763 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsk7q\" (UniqueName: \"kubernetes.io/projected/b5575c1d-f464-4425-aced-8212ec8efe84-kube-api-access-lsk7q\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:35.592451 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.592433 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2mt7\" (UniqueName: \"kubernetes.io/projected/e89141d3-30f6-4aa3-8c9e-c2fa46c8c920-kube-api-access-w2mt7\") pod \"node-exporter-4xx8m\" (UID: \"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920\") " pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.640512 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.640473 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4xx8m" Jun 09 00:13:35.656241 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:35.656201 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode89141d3_30f6_4aa3_8c9e_c2fa46c8c920.slice/crio-e3d7274ed9ca21ae4f77edad5120a938ad9cc91625627abef06ecc6d2363463a WatchSource:0}: Error finding container e3d7274ed9ca21ae4f77edad5120a938ad9cc91625627abef06ecc6d2363463a: Status 404 returned error can't find the container with id e3d7274ed9ca21ae4f77edad5120a938ad9cc91625627abef06ecc6d2363463a Jun 09 00:13:35.826135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:35.826100 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4xx8m" event={"ID":"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920","Type":"ContainerStarted","Data":"e3d7274ed9ca21ae4f77edad5120a938ad9cc91625627abef06ecc6d2363463a"} Jun 09 00:13:36.089698 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:36.089643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:36.092986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:36.092955 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5575c1d-f464-4425-aced-8212ec8efe84-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-txxtz\" (UID: \"b5575c1d-f464-4425-aced-8212ec8efe84\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:36.228975 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:36.228939 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" Jun 09 00:13:36.396144 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:36.395383 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz"] Jun 09 00:13:36.620369 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:36.620318 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5575c1d_f464_4425_aced_8212ec8efe84.slice/crio-66ada5f0fca30fda8d85a237a6413aab17debc89fc4d869a853f63c25b423c40 WatchSource:0}: Error finding container 66ada5f0fca30fda8d85a237a6413aab17debc89fc4d869a853f63c25b423c40: Status 404 returned error can't find the container with id 66ada5f0fca30fda8d85a237a6413aab17debc89fc4d869a853f63c25b423c40 Jun 09 00:13:36.832608 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:36.832572 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" event={"ID":"b5575c1d-f464-4425-aced-8212ec8efe84","Type":"ContainerStarted","Data":"66ada5f0fca30fda8d85a237a6413aab17debc89fc4d869a853f63c25b423c40"} Jun 09 00:13:36.834306 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:36.834278 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4xx8m" event={"ID":"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920","Type":"ContainerStarted","Data":"cfa5006c6d6ec8d6a2690b610b3e818f49e4cf612195050f29fc0cd86cf84dc7"} Jun 09 00:13:37.840127 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:37.840060 2550 generic.go:356] "Generic (PLEG): container finished" podID="e89141d3-30f6-4aa3-8c9e-c2fa46c8c920" containerID="cfa5006c6d6ec8d6a2690b610b3e818f49e4cf612195050f29fc0cd86cf84dc7" exitCode=0 Jun 09 00:13:37.840664 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:37.840134 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4xx8m" event={"ID":"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920","Type":"ContainerDied","Data":"cfa5006c6d6ec8d6a2690b610b3e818f49e4cf612195050f29fc0cd86cf84dc7"} Jun 09 00:13:38.596458 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:38.596238 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:38.847278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:38.847182 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4xx8m" event={"ID":"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920","Type":"ContainerStarted","Data":"0c0566d787f6a313114980831c33bf26f6ec0b6fd2b62148ac8ffce6b408e845"} Jun 09 00:13:39.738071 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.738039 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c58b74978-p5dlr"] Jun 09 00:13:39.740891 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.740874 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.743310 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.743280 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 09 00:13:39.743449 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.743292 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dm7l14ndntqmg\"" Jun 09 00:13:39.743449 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.743393 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 09 00:13:39.744032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.744008 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-b8ktv\"" Jun 09 00:13:39.744180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.744037 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 00:13:39.744180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.744048 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 09 00:13:39.749283 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.749259 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c58b74978-p5dlr"] Jun 09 00:13:39.829859 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.829768 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/10230a7c-9256-4478-8734-3aa4b1f7b4ef-metrics-server-audit-profiles\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.829859 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.829819 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10230a7c-9256-4478-8734-3aa4b1f7b4ef-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.830103 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.829895 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpf87\" (UniqueName: \"kubernetes.io/projected/10230a7c-9256-4478-8734-3aa4b1f7b4ef-kube-api-access-wpf87\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.830103 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.829947 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-client-ca-bundle\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.830103 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.829988 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-secret-metrics-server-tls\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.830214 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.830108 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-secret-metrics-server-client-certs\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.830214 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.830136 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/10230a7c-9256-4478-8734-3aa4b1f7b4ef-audit-log\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.853286 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.853254 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679bf7467b-jj7dt" event={"ID":"fced078e-da4b-404c-aec6-05b16bef129e","Type":"ContainerStarted","Data":"09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f"} Jun 09 00:13:39.855489 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.855468 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4xx8m" event={"ID":"e89141d3-30f6-4aa3-8c9e-c2fa46c8c920","Type":"ContainerStarted","Data":"c8f355f0abd72b59821cb86eca9deb75d029e253219154d859d668536abd4a76"} Jun 09 00:13:39.857628 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.857604 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" event={"ID":"b5575c1d-f464-4425-aced-8212ec8efe84","Type":"ContainerStarted","Data":"458b928db2c09976e4452732a7719b5b4701305a21ee7bbc0011efe49b6eab94"} Jun 09 00:13:39.857717 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.857655 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" event={"ID":"b5575c1d-f464-4425-aced-8212ec8efe84","Type":"ContainerStarted","Data":"739c2f3a11df44d494891f042c620c593056bbf09b8204bc789831fe4bc42e46"} Jun 09 00:13:39.857717 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.857683 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" event={"ID":"b5575c1d-f464-4425-aced-8212ec8efe84","Type":"ContainerStarted","Data":"d7931e19d43d69227455561485e4798d0ff2f8fbd836b9c2810dab769c96a0c0"} Jun 09 00:13:39.870842 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.870789 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-679bf7467b-jj7dt" podStartSLOduration=1.319764251 podStartE2EDuration="5.870770146s" totalImagesPullingTime="4.551005895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:39.868675371 +0000 UTC m=+183.093072639" watchObservedRunningTime="2026-06-09 00:13:39.870770146 +0000 UTC m=+183.095167414" Jun 09 00:13:39.889102 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.886160 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-txxtz" podStartSLOduration=2.184133122 podStartE2EDuration="4.886141603s" totalImagesPullingTime="2.702008481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:39.884688347 +0000 UTC m=+183.109085616" watchObservedRunningTime="2026-06-09 00:13:39.886141603 +0000 UTC m=+183.110538872" Jun 09 00:13:39.904348 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.904307 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4xx8m" podStartSLOduration=3.809099094 podStartE2EDuration="4.904293332s" totalImagesPullingTime="1.011489911s" totalInitContainerRuntime="83.704327ms" isStatefulPod=true podCreationTimestamp="2026-06-09 00:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:39.903188289 +0000 UTC m=+183.127585559" watchObservedRunningTime="2026-06-09 00:13:39.904293332 +0000 UTC m=+183.128690599" Jun 09 00:13:39.931485 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931461 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-secret-metrics-server-tls\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931606 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931512 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-secret-metrics-server-client-certs\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931606 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931535 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/10230a7c-9256-4478-8734-3aa4b1f7b4ef-audit-log\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931595 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/10230a7c-9256-4478-8734-3aa4b1f7b4ef-metrics-server-audit-profiles\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931649 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10230a7c-9256-4478-8734-3aa4b1f7b4ef-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931695 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpf87\" (UniqueName: \"kubernetes.io/projected/10230a7c-9256-4478-8734-3aa4b1f7b4ef-kube-api-access-wpf87\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931853 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-client-ca-bundle\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.931922 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.931903 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/10230a7c-9256-4478-8734-3aa4b1f7b4ef-audit-log\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.932503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.932477 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10230a7c-9256-4478-8734-3aa4b1f7b4ef-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.932649 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.932634 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/10230a7c-9256-4478-8734-3aa4b1f7b4ef-metrics-server-audit-profiles\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.934355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.934331 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-client-ca-bundle\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.934730 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.934708 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-secret-metrics-server-tls\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.934781 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.934747 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/10230a7c-9256-4478-8734-3aa4b1f7b4ef-secret-metrics-server-client-certs\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:39.938916 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:39.938894 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpf87\" (UniqueName: \"kubernetes.io/projected/10230a7c-9256-4478-8734-3aa4b1f7b4ef-kube-api-access-wpf87\") pod \"metrics-server-5c58b74978-p5dlr\" (UID: \"10230a7c-9256-4478-8734-3aa4b1f7b4ef\") " pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:40.051609 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.051577 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:13:40.087572 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.087482 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb"] Jun 09 00:13:40.091868 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.091841 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.094043 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.094011 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 09 00:13:40.094175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.094018 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-v2x9l\"" Jun 09 00:13:40.099465 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.099440 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb"] Jun 09 00:13:40.192750 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.192716 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c58b74978-p5dlr"] Jun 09 00:13:40.195710 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:40.195684 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10230a7c_9256_4478_8734_3aa4b1f7b4ef.slice/crio-e822167f56fdcc077ff8e5299fecdf94fd5eaa25c574a83e629ac99499bfd299 WatchSource:0}: Error finding container e822167f56fdcc077ff8e5299fecdf94fd5eaa25c574a83e629ac99499bfd299: Status 404 returned error can't find the container with id e822167f56fdcc077ff8e5299fecdf94fd5eaa25c574a83e629ac99499bfd299 Jun 09 00:13:40.234632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.234599 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7e4047ca-3d38-4a2a-93e8-038c442c2900-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8xwrb\" (UID: \"7e4047ca-3d38-4a2a-93e8-038c442c2900\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.234750 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.234651 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5klsc\" (UniqueName: \"kubernetes.io/projected/7e4047ca-3d38-4a2a-93e8-038c442c2900-kube-api-access-5klsc\") pod \"monitoring-plugin-76b68cc874-8xwrb\" (UID: \"7e4047ca-3d38-4a2a-93e8-038c442c2900\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.335516 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.335475 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5klsc\" (UniqueName: \"kubernetes.io/projected/7e4047ca-3d38-4a2a-93e8-038c442c2900-kube-api-access-5klsc\") pod \"monitoring-plugin-76b68cc874-8xwrb\" (UID: \"7e4047ca-3d38-4a2a-93e8-038c442c2900\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.335705 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.335611 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7e4047ca-3d38-4a2a-93e8-038c442c2900-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8xwrb\" (UID: \"7e4047ca-3d38-4a2a-93e8-038c442c2900\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.338337 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.338275 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7e4047ca-3d38-4a2a-93e8-038c442c2900-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8xwrb\" (UID: \"7e4047ca-3d38-4a2a-93e8-038c442c2900\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.343434 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.343410 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5klsc\" (UniqueName: \"kubernetes.io/projected/7e4047ca-3d38-4a2a-93e8-038c442c2900-kube-api-access-5klsc\") pod \"monitoring-plugin-76b68cc874-8xwrb\" (UID: \"7e4047ca-3d38-4a2a-93e8-038c442c2900\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.404911 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.404874 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:40.510505 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.510474 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-88f54cdc5-fhw7p"] Jun 09 00:13:40.514968 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.514949 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.517588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.517485 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 09 00:13:40.517588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.517533 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tztrd\"" Jun 09 00:13:40.517588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.517562 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 09 00:13:40.517819 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.517626 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 09 00:13:40.517819 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.517485 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 09 00:13:40.517819 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.517539 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 09 00:13:40.522305 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.522286 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 09 00:13:40.525917 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.525897 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-88f54cdc5-fhw7p"] Jun 09 00:13:40.543749 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.543722 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb"] Jun 09 00:13:40.547723 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:40.547699 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4047ca_3d38_4a2a_93e8_038c442c2900.slice/crio-3519ea1ba98678f44d547635b21537d76e531e1da4ac085625c85093235c3bc2 WatchSource:0}: Error finding container 3519ea1ba98678f44d547635b21537d76e531e1da4ac085625c85093235c3bc2: Status 404 returned error can't find the container with id 3519ea1ba98678f44d547635b21537d76e531e1da4ac085625c85093235c3bc2 Jun 09 00:13:40.637585 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637487 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-serving-certs-ca-bundle\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637585 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637570 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637594 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6np\" (UniqueName: \"kubernetes.io/projected/09e4639f-ab50-4369-98c3-3165b18c3cc7-kube-api-access-xg6np\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637614 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-federate-client-tls\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637670 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-metrics-client-ca\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637690 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-secret-telemeter-client\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637707 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.637773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.637724 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-telemeter-client-tls\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738666 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738702 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6np\" (UniqueName: \"kubernetes.io/projected/09e4639f-ab50-4369-98c3-3165b18c3cc7-kube-api-access-xg6np\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738723 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-federate-client-tls\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738763 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-metrics-client-ca\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738793 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-secret-telemeter-client\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738843 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-telemeter-client-tls\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.738943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.738887 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-serving-certs-ca-bundle\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.739916 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.739886 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-metrics-client-ca\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.739916 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.739908 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-serving-certs-ca-bundle\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.740100 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.739956 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09e4639f-ab50-4369-98c3-3165b18c3cc7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.741578 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.741549 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.741659 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.741643 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-telemeter-client-tls\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.741845 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.741829 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-secret-telemeter-client\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.741883 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.741859 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/09e4639f-ab50-4369-98c3-3165b18c3cc7-federate-client-tls\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.746222 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.746200 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6np\" (UniqueName: \"kubernetes.io/projected/09e4639f-ab50-4369-98c3-3165b18c3cc7-kube-api-access-xg6np\") pod \"telemeter-client-88f54cdc5-fhw7p\" (UID: \"09e4639f-ab50-4369-98c3-3165b18c3cc7\") " pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.828291 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.828254 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" Jun 09 00:13:40.863745 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.863679 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" event={"ID":"10230a7c-9256-4478-8734-3aa4b1f7b4ef","Type":"ContainerStarted","Data":"e822167f56fdcc077ff8e5299fecdf94fd5eaa25c574a83e629ac99499bfd299"} Jun 09 00:13:40.865191 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.865134 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" event={"ID":"7e4047ca-3d38-4a2a-93e8-038c442c2900","Type":"ContainerStarted","Data":"3519ea1ba98678f44d547635b21537d76e531e1da4ac085625c85093235c3bc2"} Jun 09 00:13:40.983714 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:40.983682 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-88f54cdc5-fhw7p"] Jun 09 00:13:40.987720 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:40.987693 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e4639f_ab50_4369_98c3_3165b18c3cc7.slice/crio-e78b443318a749325cf9defebb2a7febd1deb87bf612bccb1bce18b12c94e740 WatchSource:0}: Error finding container e78b443318a749325cf9defebb2a7febd1deb87bf612bccb1bce18b12c94e740: Status 404 returned error can't find the container with id e78b443318a749325cf9defebb2a7febd1deb87bf612bccb1bce18b12c94e740 Jun 09 00:13:41.526287 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.526251 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:13:41.535365 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.535332 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.537929 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.537901 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 00:13:41.538928 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.538371 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 00:13:41.538928 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.538569 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 00:13:41.538928 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.538760 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 00:13:41.540060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.539418 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 00:13:41.540060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.539645 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 00:13:41.543605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.540602 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 00:13:41.543605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.540712 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 00:13:41.543605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.540612 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 00:13:41.543605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.540892 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9r4p6\"" Jun 09 00:13:41.543605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.540964 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 00:13:41.543605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.543429 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7j2mupvp9bl1e\"" Jun 09 00:13:41.544807 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.544505 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 00:13:41.551240 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.551217 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 00:13:41.560871 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.560845 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:13:41.564180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.564156 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 00:13:41.565632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.565610 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-config-out\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.566073 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.566023 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.566812 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.566447 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 00:13:41.566999 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.566245 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.567295 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.567269 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.567488 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.567466 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.567642 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.567622 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-config\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.567779 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.567765 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.567908 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.567893 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568404 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568452 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568495 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568529 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq9d4\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-kube-api-access-qq9d4\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568571 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568603 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568658 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568683 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-web-config\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568717 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568762 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568808 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.569112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.568841 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.669831 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.669787 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.670006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.669862 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.670006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.669892 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.670006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.669921 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.670006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.669953 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq9d4\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-kube-api-access-qq9d4\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.670006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.669983 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.670006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670005 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670039 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670063 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-web-config\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670111 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670142 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670174 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670309 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.670953 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.672537 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.672671 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673116 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673162 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-config-out\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673191 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673235 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673257 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673296 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673330 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-config\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675953 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.673359 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675953 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.674497 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675953 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.674895 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.675953 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.675444 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.676527 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.676251 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-web-config\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.677601 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.676867 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.677601 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.677449 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.678193 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.677999 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.679845 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.679649 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.679845 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.679811 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-config-out\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.683763 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.683598 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq9d4\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-kube-api-access-qq9d4\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.684450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.684408 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.684450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.684441 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.684602 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.684446 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.684969 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.684948 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.686779 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.686725 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-config\") pod \"prometheus-k8s-0\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.861587 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.861200 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:13:41.872736 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:41.872683 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" event={"ID":"09e4639f-ab50-4369-98c3-3165b18c3cc7","Type":"ContainerStarted","Data":"e78b443318a749325cf9defebb2a7febd1deb87bf612bccb1bce18b12c94e740"} Jun 09 00:13:42.782371 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.782346 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:13:42.784276 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:13:42.784245 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39170b9_6f48_4a3b_8b48_d33faa092f73.slice/crio-09308bd773f421a35dfe159fee65e85a11a9b399895531a56e5e5210adba16c1 WatchSource:0}: Error finding container 09308bd773f421a35dfe159fee65e85a11a9b399895531a56e5e5210adba16c1: Status 404 returned error can't find the container with id 09308bd773f421a35dfe159fee65e85a11a9b399895531a56e5e5210adba16c1 Jun 09 00:13:42.877920 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.877823 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" event={"ID":"7e4047ca-3d38-4a2a-93e8-038c442c2900","Type":"ContainerStarted","Data":"e64ebc160cb4b76de8ad66615f0233e73c8b2836660b188e77d9aa2f70a7f480"} Jun 09 00:13:42.877920 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.877886 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:42.879430 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.879395 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"09308bd773f421a35dfe159fee65e85a11a9b399895531a56e5e5210adba16c1"} Jun 09 00:13:42.881346 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.881319 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" event={"ID":"10230a7c-9256-4478-8734-3aa4b1f7b4ef","Type":"ContainerStarted","Data":"1659f3ed823b8eb19e29f2542e4a6d838fab25830c46f1cdc0183d8671ded1f7"} Jun 09 00:13:42.883574 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.883550 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" Jun 09 00:13:42.892758 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.892710 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8xwrb" podStartSLOduration=0.81996238 podStartE2EDuration="2.892694794s" totalImagesPullingTime="2.072732414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:42.891303894 +0000 UTC m=+186.115701188" watchObservedRunningTime="2026-06-09 00:13:42.892694794 +0000 UTC m=+186.117092062" Jun 09 00:13:42.910466 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:42.910421 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" podStartSLOduration=1.487880688 podStartE2EDuration="3.910404647s" totalImagesPullingTime="2.422523959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:42.909421877 +0000 UTC m=+186.133819144" watchObservedRunningTime="2026-06-09 00:13:42.910404647 +0000 UTC m=+186.134801917" Jun 09 00:13:44.642205 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.642168 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:44.642539 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.642247 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:44.647424 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.647399 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:44.892962 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.892856 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" event={"ID":"09e4639f-ab50-4369-98c3-3165b18c3cc7","Type":"ContainerStarted","Data":"794b1563a873eb136252b24dd11dd248a782346fb25fee2e0a9fbd6eaf000b70"} Jun 09 00:13:44.892962 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.892919 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" event={"ID":"09e4639f-ab50-4369-98c3-3165b18c3cc7","Type":"ContainerStarted","Data":"54f1e7e194dbc52264a934d905df421bb8f148dc82cad37ba0eeafc6f97d647f"} Jun 09 00:13:44.892962 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.892953 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" event={"ID":"09e4639f-ab50-4369-98c3-3165b18c3cc7","Type":"ContainerStarted","Data":"d4cd9b9164416f747c38c5d2eecd4ac7af9d5e54961d2fa8fc49dd622f6f6661"} Jun 09 00:13:44.894555 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.894530 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" exitCode=0 Jun 09 00:13:44.894663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.894620 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} Jun 09 00:13:44.899039 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.899023 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:13:44.919431 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:44.919387 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-88f54cdc5-fhw7p" podStartSLOduration=1.7408906819999999 podStartE2EDuration="4.919373627s" totalImagesPullingTime="3.178482945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:44.918042492 +0000 UTC m=+188.142439809" watchObservedRunningTime="2026-06-09 00:13:44.919373627 +0000 UTC m=+188.143770895" Jun 09 00:13:50.922628 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:50.922591 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} Jun 09 00:13:50.922989 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:50.922656 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} Jun 09 00:13:53.611003 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.610937 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" podUID="bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" containerName="registry" containerID="cri-o://c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417" gracePeriod=30 Jun 09 00:13:53.846544 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.846519 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:53.896598 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896511 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tljz7\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-kube-api-access-tljz7\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896598 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896587 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-ca-trust-extracted\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896612 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-certificates\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896642 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-image-registry-private-configuration\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896695 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-bound-sa-token\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896725 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-trusted-ca\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896767 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.896820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.896797 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-installation-pull-secrets\") pod \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\" (UID: \"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4\") " Jun 09 00:13:53.897540 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.897254 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:13:53.897540 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.897286 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:13:53.899415 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.899370 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-kube-api-access-tljz7" (OuterVolumeSpecName: "kube-api-access-tljz7") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "kube-api-access-tljz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:13:53.899839 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.899815 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:13:53.899970 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.899890 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:13:53.899970 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.899948 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:13:53.900977 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.900957 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:13:53.908261 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.908235 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" (UID: "bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:13:53.941661 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.941622 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} Jun 09 00:13:53.941834 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.941706 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} Jun 09 00:13:53.941834 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.941742 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} Jun 09 00:13:53.941834 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.941780 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerStarted","Data":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} Jun 09 00:13:53.942933 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.942909 2550 generic.go:356] "Generic (PLEG): container finished" podID="bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" containerID="c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417" exitCode=0 Jun 09 00:13:53.943054 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.942947 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" Jun 09 00:13:53.943054 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.942947 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" event={"ID":"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4","Type":"ContainerDied","Data":"c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417"} Jun 09 00:13:53.943160 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.943062 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-556c4bd9f5-skfv4" event={"ID":"bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4","Type":"ContainerDied","Data":"64f6abf47c95df101753c79b45f2f9be7e9a8a651675f6145cef05d1804d5cd7"} Jun 09 00:13:53.943160 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.943101 2550 scope.go:117] "RemoveContainer" containerID="c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417" Jun 09 00:13:53.952402 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.952377 2550 scope.go:117] "RemoveContainer" containerID="c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417" Jun 09 00:13:53.952679 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:13:53.952660 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417\": container with ID starting with c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417 not found: ID does not exist" containerID="c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417" Jun 09 00:13:53.952737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.952688 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417"} err="failed to get container status \"c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417\": rpc error: code = NotFound desc = could not find container \"c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417\": container with ID starting with c0a8ab1d247c4d2687668874d02039f24d1a520f4addc93aa0365f48eb12f417 not found: ID does not exist" Jun 09 00:13:53.972225 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.972164 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.757749819 podStartE2EDuration="12.972143472s" totalImagesPullingTime="9.194722018s" totalInitContainerRuntime="19.671635ms" isStatefulPod=false podCreationTimestamp="2026-06-09 00:13:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:13:53.970189238 +0000 UTC m=+197.194586508" watchObservedRunningTime="2026-06-09 00:13:53.972143472 +0000 UTC m=+197.196540741" Jun 09 00:13:53.983973 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.983938 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-556c4bd9f5-skfv4"] Jun 09 00:13:53.987756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.987727 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-556c4bd9f5-skfv4"] Jun 09 00:13:53.998502 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998460 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tljz7\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-kube-api-access-tljz7\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998502 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998493 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-ca-trust-extracted\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998502 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998507 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-certificates\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998518 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-image-registry-private-configuration\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998527 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-bound-sa-token\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998536 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-trusted-ca\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998543 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-registry-tls\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:53.998721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:53.998554 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4-installation-pull-secrets\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:13:55.234966 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:55.234931 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" path="/var/lib/kubelet/pods/bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4/volumes" Jun 09 00:13:56.861922 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:13:56.861885 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:14:00.004331 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:00.004297 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679bf7467b-jj7dt"] Jun 09 00:14:00.052513 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:00.052475 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:14:00.052670 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:00.052538 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:14:14.022504 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:14.022472 2550 generic.go:356] "Generic (PLEG): container finished" podID="2556458f-8fa9-489e-86f4-b52a169ef0dc" containerID="7fa0a323378d8f148312bad40c40b9b85beb65b0b3ee56cddd06886078146429" exitCode=0 Jun 09 00:14:14.022873 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:14.022549 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jk8dr" event={"ID":"2556458f-8fa9-489e-86f4-b52a169ef0dc","Type":"ContainerDied","Data":"7fa0a323378d8f148312bad40c40b9b85beb65b0b3ee56cddd06886078146429"} Jun 09 00:14:14.022918 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:14.022896 2550 scope.go:117] "RemoveContainer" containerID="7fa0a323378d8f148312bad40c40b9b85beb65b0b3ee56cddd06886078146429" Jun 09 00:14:15.027635 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:15.027601 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jk8dr" event={"ID":"2556458f-8fa9-489e-86f4-b52a169ef0dc","Type":"ContainerStarted","Data":"ee7458bd444e94792a316d771419d86faf1efe82a66f593251d7cded72869afc"} Jun 09 00:14:20.058067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:20.058035 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:14:20.062047 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:20.062023 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c58b74978-p5dlr" Jun 09 00:14:25.024924 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.024852 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-679bf7467b-jj7dt" podUID="fced078e-da4b-404c-aec6-05b16bef129e" containerName="console" containerID="cri-o://09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f" gracePeriod=15 Jun 09 00:14:25.288203 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.288139 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:14:25.379657 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.379415 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjrgt\" (UniqueName: \"kubernetes.io/projected/fced078e-da4b-404c-aec6-05b16bef129e-kube-api-access-kjrgt\") pod \"fced078e-da4b-404c-aec6-05b16bef129e\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " Jun 09 00:14:25.379657 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.379500 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-serving-cert\") pod \"fced078e-da4b-404c-aec6-05b16bef129e\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " Jun 09 00:14:25.379657 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.379561 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-oauth-config\") pod \"fced078e-da4b-404c-aec6-05b16bef129e\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " Jun 09 00:14:25.379657 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.379629 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-oauth-serving-cert\") pod \"fced078e-da4b-404c-aec6-05b16bef129e\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " Jun 09 00:14:25.379657 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.379662 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-service-ca\") pod \"fced078e-da4b-404c-aec6-05b16bef129e\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " Jun 09 00:14:25.380092 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.379684 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-console-config\") pod \"fced078e-da4b-404c-aec6-05b16bef129e\" (UID: \"fced078e-da4b-404c-aec6-05b16bef129e\") " Jun 09 00:14:25.380225 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.380181 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fced078e-da4b-404c-aec6-05b16bef129e" (UID: "fced078e-da4b-404c-aec6-05b16bef129e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:14:25.380356 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.380298 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-service-ca" (OuterVolumeSpecName: "service-ca") pod "fced078e-da4b-404c-aec6-05b16bef129e" (UID: "fced078e-da4b-404c-aec6-05b16bef129e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:14:25.380437 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.380390 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-console-config" (OuterVolumeSpecName: "console-config") pod "fced078e-da4b-404c-aec6-05b16bef129e" (UID: "fced078e-da4b-404c-aec6-05b16bef129e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:14:25.382406 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.382382 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fced078e-da4b-404c-aec6-05b16bef129e" (UID: "fced078e-da4b-404c-aec6-05b16bef129e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:14:25.382565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.382547 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fced078e-da4b-404c-aec6-05b16bef129e-kube-api-access-kjrgt" (OuterVolumeSpecName: "kube-api-access-kjrgt") pod "fced078e-da4b-404c-aec6-05b16bef129e" (UID: "fced078e-da4b-404c-aec6-05b16bef129e"). InnerVolumeSpecName "kube-api-access-kjrgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:14:25.382660 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.382564 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fced078e-da4b-404c-aec6-05b16bef129e" (UID: "fced078e-da4b-404c-aec6-05b16bef129e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:14:25.481355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.481301 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-serving-cert\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:14:25.481355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.481336 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fced078e-da4b-404c-aec6-05b16bef129e-console-oauth-config\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:14:25.481355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.481350 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-oauth-serving-cert\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:14:25.481355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.481362 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-service-ca\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:14:25.481600 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.481375 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fced078e-da4b-404c-aec6-05b16bef129e-console-config\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:14:25.481600 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:25.481388 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjrgt\" (UniqueName: \"kubernetes.io/projected/fced078e-da4b-404c-aec6-05b16bef129e-kube-api-access-kjrgt\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:14:26.074725 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.074689 2550 generic.go:356] "Generic (PLEG): container finished" podID="fced078e-da4b-404c-aec6-05b16bef129e" containerID="09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f" exitCode=0 Jun 09 00:14:26.078471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.075307 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679bf7467b-jj7dt" Jun 09 00:14:26.078471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.075330 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679bf7467b-jj7dt" event={"ID":"fced078e-da4b-404c-aec6-05b16bef129e","Type":"ContainerDied","Data":"09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f"} Jun 09 00:14:26.078471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.077892 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679bf7467b-jj7dt" event={"ID":"fced078e-da4b-404c-aec6-05b16bef129e","Type":"ContainerDied","Data":"83dd697211c62adac08f1fff038d1ae0af246f62ec0e375588034e08d8b04511"} Jun 09 00:14:26.078471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.077926 2550 scope.go:117] "RemoveContainer" containerID="09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f" Jun 09 00:14:26.088162 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.088133 2550 scope.go:117] "RemoveContainer" containerID="09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f" Jun 09 00:14:26.088523 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:14:26.088496 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f\": container with ID starting with 09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f not found: ID does not exist" containerID="09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f" Jun 09 00:14:26.088626 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.088533 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f"} err="failed to get container status \"09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f\": rpc error: code = NotFound desc = could not find container \"09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f\": container with ID starting with 09492140d9b47fdc1edee2edbcd5337626de648163222345c5dc23289a42500f not found: ID does not exist" Jun 09 00:14:26.097286 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.097256 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679bf7467b-jj7dt"] Jun 09 00:14:26.101443 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:26.101417 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-679bf7467b-jj7dt"] Jun 09 00:14:27.235219 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:27.235187 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fced078e-da4b-404c-aec6-05b16bef129e" path="/var/lib/kubelet/pods/fced078e-da4b-404c-aec6-05b16bef129e/volumes" Jun 09 00:14:33.108732 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:33.108696 2550 generic.go:356] "Generic (PLEG): container finished" podID="fd0813dd-8d69-4799-9338-e9c44900f3c4" containerID="121a441ba960d43860d74e5b84d0b673bdcc6db8379d45027f0bd1b22bfc6008" exitCode=0 Jun 09 00:14:33.109158 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:33.108762 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" event={"ID":"fd0813dd-8d69-4799-9338-e9c44900f3c4","Type":"ContainerDied","Data":"121a441ba960d43860d74e5b84d0b673bdcc6db8379d45027f0bd1b22bfc6008"} Jun 09 00:14:33.109158 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:33.109037 2550 scope.go:117] "RemoveContainer" containerID="121a441ba960d43860d74e5b84d0b673bdcc6db8379d45027f0bd1b22bfc6008" Jun 09 00:14:34.115516 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:34.115478 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-phs84" event={"ID":"fd0813dd-8d69-4799-9338-e9c44900f3c4","Type":"ContainerStarted","Data":"eddc1b71fd1663fb5e3281cd683a8d25f0f7b7993d98512f1b7699e586035273"} Jun 09 00:14:41.861728 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:41.861686 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:14:41.881691 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:41.881663 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:14:42.164966 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:42.164893 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:14:48.997741 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:48.997655 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:14:49.000190 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:49.000168 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0-metrics-certs\") pod \"network-metrics-daemon-vx6qc\" (UID: \"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0\") " pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:14:49.134708 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:49.134672 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ndnwc\"" Jun 09 00:14:49.142597 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:49.142574 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vx6qc" Jun 09 00:14:49.276873 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:49.276790 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vx6qc"] Jun 09 00:14:49.279923 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:14:49.279890 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a53b44_c4c8_4e51_ae49_ee0ddda0f5f0.slice/crio-f4e7b24ed1d55fe7ae45a6a2f011325b5363c82da8f31b3926576cb644f560f1 WatchSource:0}: Error finding container f4e7b24ed1d55fe7ae45a6a2f011325b5363c82da8f31b3926576cb644f560f1: Status 404 returned error can't find the container with id f4e7b24ed1d55fe7ae45a6a2f011325b5363c82da8f31b3926576cb644f560f1 Jun 09 00:14:50.179594 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:50.179558 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vx6qc" event={"ID":"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0","Type":"ContainerStarted","Data":"f4e7b24ed1d55fe7ae45a6a2f011325b5363c82da8f31b3926576cb644f560f1"} Jun 09 00:14:51.185252 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:51.185203 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vx6qc" event={"ID":"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0","Type":"ContainerStarted","Data":"9973aaac5fd1bafea0b6f19670fe1e81a28f31a114570919935a23292aaf3c69"} Jun 09 00:14:51.185637 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:51.185306 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vx6qc" event={"ID":"c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0","Type":"ContainerStarted","Data":"37598a9670890333e4ab5a396cc4b15a6cebc2a792bbcf7b3fd60e5f879e3da9"} Jun 09 00:14:51.205042 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:51.204992 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vx6qc" podStartSLOduration=253.278111579 podStartE2EDuration="4m14.204977617s" totalImagesPullingTime="926.866038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:14:51.203510406 +0000 UTC m=+254.427907676" watchObservedRunningTime="2026-06-09 00:14:51.204977617 +0000 UTC m=+254.429374884" Jun 09 00:14:59.895472 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.895431 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:14:59.896302 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.896257 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="prometheus" containerID="cri-o://de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" gracePeriod=600 Jun 09 00:14:59.896446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.896289 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy" containerID="cri-o://30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" gracePeriod=600 Jun 09 00:14:59.896446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.896353 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" gracePeriod=600 Jun 09 00:14:59.896446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.896370 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="config-reloader" containerID="cri-o://43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" gracePeriod=600 Jun 09 00:14:59.896592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.896465 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-web" containerID="cri-o://e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" gracePeriod=600 Jun 09 00:14:59.896592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:14:59.896525 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="thanos-sidecar" containerID="cri-o://810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" gracePeriod=600 Jun 09 00:15:00.157353 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.157280 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.186093 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186051 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-config-out\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.186294 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186122 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.186294 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186151 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq9d4\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-kube-api-access-qq9d4\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.186294 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186174 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-2\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186632 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-0\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186647 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186737 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-kubelet-serving-ca-bundle\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186774 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-trusted-ca-bundle\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.186997 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-web-config\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187026 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-grpc-tls\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187062 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-db\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187141 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-config\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187170 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-kube-rbac-proxy\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187206 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-1\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187230 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-serving-certs-ca-bundle\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187255 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-metrics-client-certs\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187281 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-tls-assets\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187306 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187336 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-metrics-client-ca\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187368 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-tls\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187393 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-thanos-prometheus-http-client-file\") pod \"f39170b9-6f48-4a3b-8b48-d33faa092f73\" (UID: \"f39170b9-6f48-4a3b-8b48-d33faa092f73\") " Jun 09 00:15:00.188285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187673 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.189282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187693 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.189282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.187704 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.189282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.188294 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.189282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.188960 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.189282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.189198 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:15:00.189899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.189656 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.189899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.189714 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-config-out" (OuterVolumeSpecName: "config-out") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:15:00.189899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.189784 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.189899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.189859 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-kube-api-access-qq9d4" (OuterVolumeSpecName: "kube-api-access-qq9d4") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "kube-api-access-qq9d4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:15:00.190770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.190726 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 00:15:00.194222 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.194017 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-config" (OuterVolumeSpecName: "config") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.196845 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.196791 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.197131 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.197020 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.197219 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.197156 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.197278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.197211 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.197278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.197238 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:15:00.197375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.197296 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.197375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.197315 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.211751 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.211716 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-web-config" (OuterVolumeSpecName: "web-config") pod "f39170b9-6f48-4a3b-8b48-d33faa092f73" (UID: "f39170b9-6f48-4a3b-8b48-d33faa092f73"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:15:00.224723 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224688 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" exitCode=0 Jun 09 00:15:00.224723 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224721 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" exitCode=0 Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224730 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" exitCode=0 Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224739 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" exitCode=0 Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224746 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" exitCode=0 Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224754 2550 generic.go:356] "Generic (PLEG): container finished" podID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" exitCode=0 Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224778 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224787 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224876 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} Jun 09 00:15:00.224932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224924 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} Jun 09 00:15:00.225349 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224956 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.225349 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.224963 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} Jun 09 00:15:00.225349 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.225007 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} Jun 09 00:15:00.225349 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.225052 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} Jun 09 00:15:00.225349 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.225112 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f39170b9-6f48-4a3b-8b48-d33faa092f73","Type":"ContainerDied","Data":"09308bd773f421a35dfe159fee65e85a11a9b399895531a56e5e5210adba16c1"} Jun 09 00:15:00.233802 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.233600 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.242484 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.242463 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.248562 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.248535 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:15:00.252653 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.252628 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.253137 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.253106 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:15:00.260333 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.260310 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.267580 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.267563 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.275164 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.275143 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.282450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.282420 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.282768 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.282743 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.282845 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.282782 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} err="failed to get container status \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" Jun 09 00:15:00.282845 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.282806 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.283071 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.283057 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.283153 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283100 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} err="failed to get container status \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" Jun 09 00:15:00.283153 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283114 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.283352 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.283338 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.283446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283354 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} err="failed to get container status \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" Jun 09 00:15:00.283446 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283367 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.283594 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.283573 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.283675 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283605 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} err="failed to get container status \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" Jun 09 00:15:00.283675 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283629 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.283889 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.283875 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.283932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283891 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} err="failed to get container status \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" Jun 09 00:15:00.283932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.283902 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.284159 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.284142 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.284216 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284161 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} err="failed to get container status \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" Jun 09 00:15:00.284216 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284181 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.284389 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:15:00.284371 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.284458 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284405 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} err="failed to get container status \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" Jun 09 00:15:00.284458 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284427 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.284458 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284432 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:15:00.284659 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284632 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} err="failed to get container status \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" Jun 09 00:15:00.284718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284659 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.284718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284711 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fced078e-da4b-404c-aec6-05b16bef129e" containerName="console" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284723 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fced078e-da4b-404c-aec6-05b16bef129e" containerName="console" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284732 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="init-config-reloader" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284737 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="init-config-reloader" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284754 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="config-reloader" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284763 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="config-reloader" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284772 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284781 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284791 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="prometheus" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284796 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="prometheus" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284807 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-web" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284813 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-web" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284821 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" containerName="registry" Jun 09 00:15:00.284828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284829 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" containerName="registry" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284840 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="thanos-sidecar" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284848 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="thanos-sidecar" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284854 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-thanos" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284859 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-thanos" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284876 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} err="failed to get container status \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284889 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284914 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fced078e-da4b-404c-aec6-05b16bef129e" containerName="console" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284921 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd06ca0f-5c26-47d7-ae03-1a1980e0f7b4" containerName="registry" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284927 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284936 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="prometheus" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284942 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="thanos-sidecar" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284947 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="config-reloader" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284952 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-web" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.284958 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" containerName="kube-rbac-proxy-thanos" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285108 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} err="failed to get container status \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" Jun 09 00:15:00.285267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285123 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285295 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} err="failed to get container status \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285307 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285464 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} err="failed to get container status \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285476 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285720 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} err="failed to get container status \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285745 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.285979 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} err="failed to get container status \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" Jun 09 00:15:00.285997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.286003 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.286436 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.286265 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} err="failed to get container status \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" Jun 09 00:15:00.286436 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.286304 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.286738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.286716 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} err="failed to get container status \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" Jun 09 00:15:00.286837 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.286739 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.286998 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.286971 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} err="failed to get container status \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" Jun 09 00:15:00.287061 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287002 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.287308 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287289 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} err="failed to get container status \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" Jun 09 00:15:00.287362 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287308 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.287605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287588 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} err="failed to get container status \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" Jun 09 00:15:00.287656 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287607 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.287835 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287815 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} err="failed to get container status \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" Jun 09 00:15:00.287835 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.287835 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.288104 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288062 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} err="failed to get container status \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" Jun 09 00:15:00.288104 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288100 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.288217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288151 2550 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-config\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288171 2550 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-kube-rbac-proxy\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288185 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288202 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288215 2550 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-metrics-client-certs\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288228 2550 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-tls-assets\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288241 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288255 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-metrics-client-ca\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288267 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288280 2550 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288293 2550 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-config-out\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288305 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288318 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq9d4\" (UniqueName: \"kubernetes.io/projected/f39170b9-6f48-4a3b-8b48-d33faa092f73-kube-api-access-qq9d4\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288331 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288335 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} err="failed to get container status \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288359 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288344 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288402 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288420 2550 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-web-config\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288430 2550 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f39170b9-6f48-4a3b-8b48-d33faa092f73-secret-grpc-tls\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.288452 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288439 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f39170b9-6f48-4a3b-8b48-d33faa092f73-prometheus-k8s-db\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:15:00.289096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288604 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} err="failed to get container status \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" Jun 09 00:15:00.289096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288619 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.289096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288819 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} err="failed to get container status \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" Jun 09 00:15:00.289096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.288835 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.289096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289031 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} err="failed to get container status \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" Jun 09 00:15:00.289096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289054 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.289350 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289290 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} err="failed to get container status \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" Jun 09 00:15:00.289350 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289303 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.289476 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289460 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} err="failed to get container status \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" Jun 09 00:15:00.289476 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289476 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.289675 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289659 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} err="failed to get container status \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" Jun 09 00:15:00.289732 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289675 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.289921 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289891 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} err="failed to get container status \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" Jun 09 00:15:00.289974 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.289922 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.290218 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290200 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} err="failed to get container status \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" Jun 09 00:15:00.290300 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290217 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.290499 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290480 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} err="failed to get container status \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" Jun 09 00:15:00.290559 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290500 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.290690 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290672 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.290785 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290693 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} err="failed to get container status \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" Jun 09 00:15:00.290785 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290708 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.291034 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290937 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} err="failed to get container status \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" Jun 09 00:15:00.291034 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.290964 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.291760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.291694 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} err="failed to get container status \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" Jun 09 00:15:00.291760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.291761 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.292176 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} err="failed to get container status \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.292205 2550 scope.go:117] "RemoveContainer" containerID="a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.292989 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.293403 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.293441 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.293830 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.293842 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.293861 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294274 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4"} err="failed to get container status \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": rpc error: code = NotFound desc = could not find container \"a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4\": container with ID starting with a48f209d7a2dc085ae9b55803bfdec1d0f73d497462d30e3b048d9f9b8f237d4 not found: ID does not exist" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294295 2550 scope.go:117] "RemoveContainer" containerID="30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294502 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 00:15:00.294695 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294553 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 00:15:00.295311 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294757 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 00:15:00.295311 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294891 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 00:15:00.295311 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.294921 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 00:15:00.295311 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.295267 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9r4p6\"" Jun 09 00:15:00.295483 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.295348 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82"} err="failed to get container status \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": rpc error: code = NotFound desc = could not find container \"30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82\": container with ID starting with 30ea8e25e79dbf9326e60c3e9948832a9db25838eaf3d229ee2971dd36a07a82 not found: ID does not exist" Jun 09 00:15:00.295483 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.295371 2550 scope.go:117] "RemoveContainer" containerID="e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.296986 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36"} err="failed to get container status \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": rpc error: code = NotFound desc = could not find container \"e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36\": container with ID starting with e14281f6c5942fe4b755c6400f6c58e0d92c7a7a9d4830532ae70d5f9d40fd36 not found: ID does not exist" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.297013 2550 scope.go:117] "RemoveContainer" containerID="810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.297383 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7j2mupvp9bl1e\"" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.297387 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.298827 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8"} err="failed to get container status \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": rpc error: code = NotFound desc = could not find container \"810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8\": container with ID starting with 810d3565e5f86fe723ea8bb27fdaed6c3e6c3fccaaf1ae6f4e61a7750cdf7ff8 not found: ID does not exist" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.298850 2550 scope.go:117] "RemoveContainer" containerID="43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.299138 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9"} err="failed to get container status \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": rpc error: code = NotFound desc = could not find container \"43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9\": container with ID starting with 43e07a8b326d28aba155c1c6e0a6fd01b72d3b91be01154bbabde24a15923ec9 not found: ID does not exist" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.299161 2550 scope.go:117] "RemoveContainer" containerID="de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.299431 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8"} err="failed to get container status \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": rpc error: code = NotFound desc = could not find container \"de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8\": container with ID starting with de6bb42679063a3698bc463be077a001162ad550b6fc0dcf17e332214695bbf8 not found: ID does not exist" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.299453 2550 scope.go:117] "RemoveContainer" containerID="1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.299954 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 00:15:00.300140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.300066 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855"} err="failed to get container status \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": rpc error: code = NotFound desc = could not find container \"1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855\": container with ID starting with 1b301d8a724b5c3b5df49c8680a5704a597e492f3063fc9b4e48867cac60b855 not found: ID does not exist" Jun 09 00:15:00.304974 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.302791 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 00:15:00.304974 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.304294 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:15:00.389651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389615 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.389651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389653 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.389880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389679 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-config\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.389880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389733 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.389880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389818 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.389880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389836 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.389880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389851 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389882 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389908 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389925 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389950 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.389989 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgpm2\" (UniqueName: \"kubernetes.io/projected/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-kube-api-access-hgpm2\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390016 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390038 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390063 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390054 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390068 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390122 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390151 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390169 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.390355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.390214 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491464 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491423 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491471 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491499 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491521 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491552 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491580 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491603 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491631 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-config\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.491667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491651 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491709 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491732 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491771 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491800 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491830 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491872 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491903 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgpm2\" (UniqueName: \"kubernetes.io/projected/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-kube-api-access-hgpm2\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491933 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491963 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.491986 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492722 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.492367 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492722 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.492468 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.492828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.492733 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.493222 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.493198 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.494513 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.494484 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495117 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.494757 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495117 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.494788 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495117 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.494825 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495117 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.494951 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-config\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495887 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.495527 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495887 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.495651 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495887 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.495686 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495887 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.495806 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.495887 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.495829 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.496915 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.496887 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.497836 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.497815 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.498041 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.498018 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.498210 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.498190 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.498526 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.498506 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.501457 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.501436 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgpm2\" (UniqueName: \"kubernetes.io/projected/7e69f0b3-15c3-4d0f-bc64-7c03006a275c-kube-api-access-hgpm2\") pod \"prometheus-k8s-0\" (UID: \"7e69f0b3-15c3-4d0f-bc64-7c03006a275c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.606451 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.606412 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:00.749323 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:00.749247 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 00:15:00.752738 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:15:00.752715 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e69f0b3_15c3_4d0f_bc64_7c03006a275c.slice/crio-1dbc99bfcf3e5f862beddf3b17c53a9a142dda097e00ed6820b2e41551bf8df7 WatchSource:0}: Error finding container 1dbc99bfcf3e5f862beddf3b17c53a9a142dda097e00ed6820b2e41551bf8df7: Status 404 returned error can't find the container with id 1dbc99bfcf3e5f862beddf3b17c53a9a142dda097e00ed6820b2e41551bf8df7 Jun 09 00:15:01.231394 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:01.231359 2550 generic.go:356] "Generic (PLEG): container finished" podID="7e69f0b3-15c3-4d0f-bc64-7c03006a275c" containerID="bc515f24807004c8d4f535e677edd114b13989b267a907b5544b1cbee41ded67" exitCode=0 Jun 09 00:15:01.234971 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:01.234941 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39170b9-6f48-4a3b-8b48-d33faa092f73" path="/var/lib/kubelet/pods/f39170b9-6f48-4a3b-8b48-d33faa092f73/volumes" Jun 09 00:15:01.235731 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:01.235704 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerDied","Data":"bc515f24807004c8d4f535e677edd114b13989b267a907b5544b1cbee41ded67"} Jun 09 00:15:01.235866 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:01.235774 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"1dbc99bfcf3e5f862beddf3b17c53a9a142dda097e00ed6820b2e41551bf8df7"} Jun 09 00:15:02.238866 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.238833 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"a69906f5fa37d9c3c184b10a7d5295b0995cf51d19f85af73da155b7d79a4135"} Jun 09 00:15:02.239266 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.238902 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"da9264080c2d32396f0d4a31e973aefa7d5b0b0b15788c4079cb5cc15e40cdee"} Jun 09 00:15:02.239266 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.238930 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"a7f7ffe764b19ed1b91c5b3d16f273c06a43f3c6df00360320a2412853d39e1f"} Jun 09 00:15:02.239266 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.238964 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"c627e6cb0fb22305d852137e9f520f6858443dfb03ee6ecdc40ec43cfd1b5994"} Jun 09 00:15:02.239266 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.238993 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"c7b7deec34ecb5b3f5295d6a8fa86d692e5a457653bc30785253bd962db047a6"} Jun 09 00:15:02.239266 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.239016 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e69f0b3-15c3-4d0f-bc64-7c03006a275c","Type":"ContainerStarted","Data":"645acd6a45661bd59fd77597b9101fe03c8c1dca2357742b3eb4e7519c170033"} Jun 09 00:15:02.266341 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:02.266287 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.247469561 podStartE2EDuration="2.266271909s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.802348ms" isStatefulPod=false podCreationTimestamp="2026-06-09 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:15:02.263965886 +0000 UTC m=+265.488363178" watchObservedRunningTime="2026-06-09 00:15:02.266271909 +0000 UTC m=+265.490669244" Jun 09 00:15:05.607526 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:05.607481 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:15:37.174299 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:15:37.174273 2550 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 00:16:00.607545 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:00.607507 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:16:00.623897 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:00.623871 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:16:01.482067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:01.482036 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 00:16:10.820949 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.820912 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q"] Jun 09 00:16:10.824468 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.824446 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:10.826756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.826731 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 09 00:16:10.827617 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.827592 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 09 00:16:10.827727 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.827640 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gzf85\"" Jun 09 00:16:10.833333 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.833308 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q"] Jun 09 00:16:10.907930 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.907895 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-p7j7q\" (UID: \"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:10.908101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:10.907964 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9x64\" (UniqueName: \"kubernetes.io/projected/a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01-kube-api-access-z9x64\") pod \"cert-manager-webhook-7dc76cbb94-p7j7q\" (UID: \"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:11.009295 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.009256 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-p7j7q\" (UID: \"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:11.009455 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.009318 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9x64\" (UniqueName: \"kubernetes.io/projected/a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01-kube-api-access-z9x64\") pod \"cert-manager-webhook-7dc76cbb94-p7j7q\" (UID: \"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:11.016882 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.016846 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-p7j7q\" (UID: \"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:11.016990 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.016903 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9x64\" (UniqueName: \"kubernetes.io/projected/a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01-kube-api-access-z9x64\") pod \"cert-manager-webhook-7dc76cbb94-p7j7q\" (UID: \"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:11.148663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.148574 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:11.282398 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.282372 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q"] Jun 09 00:16:11.285013 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:16:11.284989 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7aff9f0_b552_48e6_88a2_1ddc0cf2ae01.slice/crio-57036adfcb2f8014605d023e075d8dc3f5f37e3d137e58b02407eb0c9656d30a WatchSource:0}: Error finding container 57036adfcb2f8014605d023e075d8dc3f5f37e3d137e58b02407eb0c9656d30a: Status 404 returned error can't find the container with id 57036adfcb2f8014605d023e075d8dc3f5f37e3d137e58b02407eb0c9656d30a Jun 09 00:16:11.287109 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.287066 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 00:16:11.506742 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:11.506705 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" event={"ID":"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01","Type":"ContainerStarted","Data":"57036adfcb2f8014605d023e075d8dc3f5f37e3d137e58b02407eb0c9656d30a"} Jun 09 00:16:14.522362 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:14.522279 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" event={"ID":"a7aff9f0-b552-48e6-88a2-1ddc0cf2ae01","Type":"ContainerStarted","Data":"95894ef473be7261a505a341f368a19773641bd678575f9d1809fb784b6f7d27"} Jun 09 00:16:14.522362 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:14.522328 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:14.537273 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:14.537223 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" podStartSLOduration=1.433247668 podStartE2EDuration="4.537209395s" totalImagesPullingTime="3.103961727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:16:14.536603492 +0000 UTC m=+337.761000761" watchObservedRunningTime="2026-06-09 00:16:14.537209395 +0000 UTC m=+337.761606662" Jun 09 00:16:20.528707 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:20.528677 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-p7j7q" Jun 09 00:16:43.703621 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.703581 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5"] Jun 09 00:16:43.710448 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.710423 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.714370 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.714338 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 00:16:43.714487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.714451 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ncvpp\"" Jun 09 00:16:43.715312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.715292 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 00:16:43.715402 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.715368 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 00:16:43.715465 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.715412 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 00:16:43.734118 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.734069 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5"] Jun 09 00:16:43.804170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.804130 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d915f03-acb8-496b-ae92-506da5515857-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.804170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.804172 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5nd5\" (UniqueName: \"kubernetes.io/projected/9d915f03-acb8-496b-ae92-506da5515857-kube-api-access-v5nd5\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.804395 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.804235 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d915f03-acb8-496b-ae92-506da5515857-tmp\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.804395 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.804292 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d915f03-acb8-496b-ae92-506da5515857-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.804395 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.804356 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d915f03-acb8-496b-ae92-506da5515857-manifests\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.905252 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.905208 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d915f03-acb8-496b-ae92-506da5515857-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.905252 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.905254 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5nd5\" (UniqueName: \"kubernetes.io/projected/9d915f03-acb8-496b-ae92-506da5515857-kube-api-access-v5nd5\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.905505 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.905296 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d915f03-acb8-496b-ae92-506da5515857-tmp\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.905505 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.905345 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d915f03-acb8-496b-ae92-506da5515857-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.905505 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.905382 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d915f03-acb8-496b-ae92-506da5515857-manifests\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.905869 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.905833 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d915f03-acb8-496b-ae92-506da5515857-manifests\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.907872 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.907842 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d915f03-acb8-496b-ae92-506da5515857-tmp\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.907986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.907921 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d915f03-acb8-496b-ae92-506da5515857-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.908067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.908004 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d915f03-acb8-496b-ae92-506da5515857-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:43.913204 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:43.913182 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5nd5\" (UniqueName: \"kubernetes.io/projected/9d915f03-acb8-496b-ae92-506da5515857-kube-api-access-v5nd5\") pod \"opendatahub-operator-controller-manager-b8d497bc9-nzjv5\" (UID: \"9d915f03-acb8-496b-ae92-506da5515857\") " pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:44.021260 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:44.021225 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:44.168348 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:44.168323 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5"] Jun 09 00:16:44.171012 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:16:44.170980 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d915f03_acb8_496b_ae92_506da5515857.slice/crio-e4335582f3256ffe543b8b22f61fcb7ace114b1bd15c62e6a53c1280ecdb8273 WatchSource:0}: Error finding container e4335582f3256ffe543b8b22f61fcb7ace114b1bd15c62e6a53c1280ecdb8273: Status 404 returned error can't find the container with id e4335582f3256ffe543b8b22f61fcb7ace114b1bd15c62e6a53c1280ecdb8273 Jun 09 00:16:44.644651 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:44.644609 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" event={"ID":"9d915f03-acb8-496b-ae92-506da5515857","Type":"ContainerStarted","Data":"e4335582f3256ffe543b8b22f61fcb7ace114b1bd15c62e6a53c1280ecdb8273"} Jun 09 00:16:47.661171 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:47.661136 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" event={"ID":"9d915f03-acb8-496b-ae92-506da5515857","Type":"ContainerStarted","Data":"6c215505f86075d4c1986ce7f91731cd4591d530ec499f3ea36d81f9a81d668d"} Jun 09 00:16:48.667143 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:48.667105 2550 generic.go:356] "Generic (PLEG): container finished" podID="9d915f03-acb8-496b-ae92-506da5515857" containerID="6c215505f86075d4c1986ce7f91731cd4591d530ec499f3ea36d81f9a81d668d" exitCode=0 Jun 09 00:16:48.667521 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:48.667185 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" event={"ID":"9d915f03-acb8-496b-ae92-506da5515857","Type":"ContainerDied","Data":"6c215505f86075d4c1986ce7f91731cd4591d530ec499f3ea36d81f9a81d668d"} Jun 09 00:16:49.674232 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:49.674192 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" event={"ID":"9d915f03-acb8-496b-ae92-506da5515857","Type":"ContainerStarted","Data":"c6573859fc8de3618a70cce7f308735296e286c877f4ca66bdb2e2c7257fad81"} Jun 09 00:16:49.674602 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:49.674306 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:16:49.697339 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:16:49.697289 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" podStartSLOduration=2.449691997 podStartE2EDuration="6.697272999s" totalImagesPullingTime="3.244409429s" totalInitContainerRuntime="1.003171573s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:16:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:16:49.695669525 +0000 UTC m=+372.920066797" watchObservedRunningTime="2026-06-09 00:16:49.697272999 +0000 UTC m=+372.921670266" Jun 09 00:17:00.680682 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:00.680654 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b8d497bc9-nzjv5" Jun 09 00:17:02.856808 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.856772 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz"] Jun 09 00:17:02.866999 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.866966 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:02.869927 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.869899 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz"] Jun 09 00:17:02.871234 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.870834 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 09 00:17:02.871234 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.870858 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 09 00:17:02.871234 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.870877 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 09 00:17:02.871234 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.871025 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 09 00:17:02.871234 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.871054 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 09 00:17:02.871234 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.871132 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-s67zs\"" Jun 09 00:17:02.976301 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.976253 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl6kz\" (UniqueName: \"kubernetes.io/projected/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-kube-api-access-kl6kz\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:02.976468 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.976316 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-manager-config\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:02.976468 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.976417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-cert\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:02.976538 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:02.976469 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-metrics-cert\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.077332 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.077291 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-cert\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.077332 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.077341 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-metrics-cert\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.077605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.077401 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl6kz\" (UniqueName: \"kubernetes.io/projected/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-kube-api-access-kl6kz\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.077605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.077435 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-manager-config\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.078372 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.078343 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-manager-config\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.080014 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.079993 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-metrics-cert\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.080128 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.080043 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-cert\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.085422 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.085397 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl6kz\" (UniqueName: \"kubernetes.io/projected/5fc22ee6-96ae-419c-9472-cd8c170e0bfd-kube-api-access-kl6kz\") pod \"lws-controller-manager-844bd95cf6-fpmxz\" (UID: \"5fc22ee6-96ae-419c-9472-cd8c170e0bfd\") " pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.177881 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.177785 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:03.325570 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.325535 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz"] Jun 09 00:17:03.329182 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:17:03.329150 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc22ee6_96ae_419c_9472_cd8c170e0bfd.slice/crio-5ac79ea910a62b5c1b0b33a71ceaa08e08d7cfca361a30cbbaa65d6a42cd5211 WatchSource:0}: Error finding container 5ac79ea910a62b5c1b0b33a71ceaa08e08d7cfca361a30cbbaa65d6a42cd5211: Status 404 returned error can't find the container with id 5ac79ea910a62b5c1b0b33a71ceaa08e08d7cfca361a30cbbaa65d6a42cd5211 Jun 09 00:17:03.742730 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:03.742697 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" event={"ID":"5fc22ee6-96ae-419c-9472-cd8c170e0bfd","Type":"ContainerStarted","Data":"5ac79ea910a62b5c1b0b33a71ceaa08e08d7cfca361a30cbbaa65d6a42cd5211"} Jun 09 00:17:06.761464 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:06.761421 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" event={"ID":"5fc22ee6-96ae-419c-9472-cd8c170e0bfd","Type":"ContainerStarted","Data":"ceee53e6e30322345c6bee76b54da3a0814058a650485711a30def1d33089c75"} Jun 09 00:17:06.761817 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:06.761564 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:17.768715 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:17.768680 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" Jun 09 00:17:17.784005 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:17.783949 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-844bd95cf6-fpmxz" podStartSLOduration=12.900603175 podStartE2EDuration="15.783934026s" totalImagesPullingTime="2.883330851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:17:06.77668758 +0000 UTC m=+390.001084849" watchObservedRunningTime="2026-06-09 00:17:17.783934026 +0000 UTC m=+401.008331295" Jun 09 00:17:32.870924 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.870889 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-77774b9544-tlt67"] Jun 09 00:17:32.874614 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.874591 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:32.877135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.877108 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 09 00:17:32.877279 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.877145 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kb6fp\"" Jun 09 00:17:32.882444 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.882370 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-77774b9544-tlt67"] Jun 09 00:17:32.955801 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.955764 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706-redirect-config\") pod \"dashboard-redirect-77774b9544-tlt67\" (UID: \"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706\") " pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:32.955976 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:32.955830 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z49j\" (UniqueName: \"kubernetes.io/projected/c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706-kube-api-access-9z49j\") pod \"dashboard-redirect-77774b9544-tlt67\" (UID: \"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706\") " pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:33.056282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.056249 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706-redirect-config\") pod \"dashboard-redirect-77774b9544-tlt67\" (UID: \"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706\") " pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:33.056445 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.056318 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z49j\" (UniqueName: \"kubernetes.io/projected/c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706-kube-api-access-9z49j\") pod \"dashboard-redirect-77774b9544-tlt67\" (UID: \"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706\") " pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:33.056961 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.056943 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706-redirect-config\") pod \"dashboard-redirect-77774b9544-tlt67\" (UID: \"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706\") " pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:33.065003 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.064969 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z49j\" (UniqueName: \"kubernetes.io/projected/c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706-kube-api-access-9z49j\") pod \"dashboard-redirect-77774b9544-tlt67\" (UID: \"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706\") " pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:33.187342 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.187257 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-77774b9544-tlt67" Jun 09 00:17:33.351056 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.351028 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-77774b9544-tlt67"] Jun 09 00:17:33.353221 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:17:33.353196 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f08b8f_c24e_47e6_9ee1_dd8cfd4e9706.slice/crio-e812ff144eea5796f065899cdc04cc136813cd27304825a8a3170fd64f3cc97f WatchSource:0}: Error finding container e812ff144eea5796f065899cdc04cc136813cd27304825a8a3170fd64f3cc97f: Status 404 returned error can't find the container with id e812ff144eea5796f065899cdc04cc136813cd27304825a8a3170fd64f3cc97f Jun 09 00:17:33.887888 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:33.887849 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-77774b9544-tlt67" event={"ID":"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706","Type":"ContainerStarted","Data":"e812ff144eea5796f065899cdc04cc136813cd27304825a8a3170fd64f3cc97f"} Jun 09 00:17:35.898449 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:35.898375 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-77774b9544-tlt67" event={"ID":"c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706","Type":"ContainerStarted","Data":"1f583ee41814e8f9359ee4c335316d842be1133a4a69e578f475d17547a6e155"} Jun 09 00:17:35.913042 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:35.912967 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-77774b9544-tlt67" podStartSLOduration=2.183978006 podStartE2EDuration="3.912952498s" totalImagesPullingTime="1.728974492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:17:35.912350308 +0000 UTC m=+419.136747578" watchObservedRunningTime="2026-06-09 00:17:35.912952498 +0000 UTC m=+419.137349765" Jun 09 00:17:51.744115 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.744020 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh"] Jun 09 00:17:51.747381 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.747355 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.749805 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.749783 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 09 00:17:51.749919 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.749806 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6jm9b\"" Jun 09 00:17:51.750095 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.750058 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 00:17:51.750178 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.750069 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 00:17:51.759170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.759150 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh"] Jun 09 00:17:51.922254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922216 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922302 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922342 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922379 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922402 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922427 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh84v\" (UniqueName: \"kubernetes.io/projected/e467d419-13df-4ca3-a5c0-2c520ed5ac73-kube-api-access-xh84v\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922492 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922543 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:51.923145 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:51.922646 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023287 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023197 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023287 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023244 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023287 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023277 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023345 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023402 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023461 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023501 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023527 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023588 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023554 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh84v\" (UniqueName: \"kubernetes.io/projected/e467d419-13df-4ca3-a5c0-2c520ed5ac73-kube-api-access-xh84v\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.023866 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.023701 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.024128 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.024103 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.024413 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.024386 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.024537 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.024420 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.024604 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.024569 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.025946 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.025921 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.026865 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.026845 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.031906 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.031880 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e467d419-13df-4ca3-a5c0-2c520ed5ac73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.032571 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.032550 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh84v\" (UniqueName: \"kubernetes.io/projected/e467d419-13df-4ca3-a5c0-2c520ed5ac73-kube-api-access-xh84v\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc5pcjh\" (UID: \"e467d419-13df-4ca3-a5c0-2c520ed5ac73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.061161 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.061128 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:17:52.215101 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.215050 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh"] Jun 09 00:17:52.217900 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:17:52.217867 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode467d419_13df_4ca3_a5c0_2c520ed5ac73.slice/crio-efb8f28be3dff63acf885f604a5de3bc2ccdd0241aab7fbf12af08ef8d9d9bc2 WatchSource:0}: Error finding container efb8f28be3dff63acf885f604a5de3bc2ccdd0241aab7fbf12af08ef8d9d9bc2: Status 404 returned error can't find the container with id efb8f28be3dff63acf885f604a5de3bc2ccdd0241aab7fbf12af08ef8d9d9bc2 Jun 09 00:17:52.975926 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:17:52.975889 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" event={"ID":"e467d419-13df-4ca3-a5c0-2c520ed5ac73","Type":"ContainerStarted","Data":"efb8f28be3dff63acf885f604a5de3bc2ccdd0241aab7fbf12af08ef8d9d9bc2"} Jun 09 00:18:03.999568 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:03.999534 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qmjfx"] Jun 09 00:18:04.004718 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.004692 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:04.007471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.007433 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 00:18:04.007471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.007458 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 00:18:04.008273 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.008253 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-smdbj\"" Jun 09 00:18:04.019836 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.019802 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qmjfx"] Jun 09 00:18:04.034444 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.034413 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd7j8\" (UniqueName: \"kubernetes.io/projected/fea0a509-d31a-4035-95c1-496d0e018e5f-kube-api-access-fd7j8\") pod \"kuadrant-operator-catalog-qmjfx\" (UID: \"fea0a509-d31a-4035-95c1-496d0e018e5f\") " pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:04.135134 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.135066 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd7j8\" (UniqueName: \"kubernetes.io/projected/fea0a509-d31a-4035-95c1-496d0e018e5f-kube-api-access-fd7j8\") pod \"kuadrant-operator-catalog-qmjfx\" (UID: \"fea0a509-d31a-4035-95c1-496d0e018e5f\") " pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:04.148339 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.148297 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd7j8\" (UniqueName: \"kubernetes.io/projected/fea0a509-d31a-4035-95c1-496d0e018e5f-kube-api-access-fd7j8\") pod \"kuadrant-operator-catalog-qmjfx\" (UID: \"fea0a509-d31a-4035-95c1-496d0e018e5f\") " pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:04.326979 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.326886 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:04.365008 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.364970 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qmjfx"] Jun 09 00:18:04.461748 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.461718 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qmjfx"] Jun 09 00:18:04.465608 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:04.465578 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea0a509_d31a_4035_95c1_496d0e018e5f.slice/crio-2a8f87cc32803356a29ea1c919c5f786911855900d7d946e789ac905637f147c WatchSource:0}: Error finding container 2a8f87cc32803356a29ea1c919c5f786911855900d7d946e789ac905637f147c: Status 404 returned error can't find the container with id 2a8f87cc32803356a29ea1c919c5f786911855900d7d946e789ac905637f147c Jun 09 00:18:04.573727 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.573684 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wj4g8"] Jun 09 00:18:04.578306 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.578244 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:04.583430 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.583404 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wj4g8"] Jun 09 00:18:04.640041 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.640005 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkcf4\" (UniqueName: \"kubernetes.io/projected/a57d7aca-4044-4927-a7ca-177025e9f57f-kube-api-access-vkcf4\") pod \"kuadrant-operator-catalog-wj4g8\" (UID: \"a57d7aca-4044-4927-a7ca-177025e9f57f\") " pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:04.741001 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.740965 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkcf4\" (UniqueName: \"kubernetes.io/projected/a57d7aca-4044-4927-a7ca-177025e9f57f-kube-api-access-vkcf4\") pod \"kuadrant-operator-catalog-wj4g8\" (UID: \"a57d7aca-4044-4927-a7ca-177025e9f57f\") " pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:04.749267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.749243 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkcf4\" (UniqueName: \"kubernetes.io/projected/a57d7aca-4044-4927-a7ca-177025e9f57f-kube-api-access-vkcf4\") pod \"kuadrant-operator-catalog-wj4g8\" (UID: \"a57d7aca-4044-4927-a7ca-177025e9f57f\") " pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:04.889847 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:04.889759 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:05.021226 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:05.021189 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wj4g8"] Jun 09 00:18:05.024632 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:05.024567 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57d7aca_4044_4927_a7ca_177025e9f57f.slice/crio-8d03f31144c480c2d8f5c1e6c9816e5b857dd55122c8777a364fff8958d90382 WatchSource:0}: Error finding container 8d03f31144c480c2d8f5c1e6c9816e5b857dd55122c8777a364fff8958d90382: Status 404 returned error can't find the container with id 8d03f31144c480c2d8f5c1e6c9816e5b857dd55122c8777a364fff8958d90382 Jun 09 00:18:05.033764 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:05.033736 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" event={"ID":"fea0a509-d31a-4035-95c1-496d0e018e5f","Type":"ContainerStarted","Data":"2a8f87cc32803356a29ea1c919c5f786911855900d7d946e789ac905637f147c"} Jun 09 00:18:05.034796 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:05.034773 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" event={"ID":"a57d7aca-4044-4927-a7ca-177025e9f57f","Type":"ContainerStarted","Data":"8d03f31144c480c2d8f5c1e6c9816e5b857dd55122c8777a364fff8958d90382"} Jun 09 00:18:07.044974 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.044937 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" event={"ID":"fea0a509-d31a-4035-95c1-496d0e018e5f","Type":"ContainerStarted","Data":"171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538"} Jun 09 00:18:07.044974 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.044963 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" podUID="fea0a509-d31a-4035-95c1-496d0e018e5f" containerName="registry-server" containerID="cri-o://171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538" gracePeriod=2 Jun 09 00:18:07.046668 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.046638 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" event={"ID":"a57d7aca-4044-4927-a7ca-177025e9f57f","Type":"ContainerStarted","Data":"d6d07bbf136cdda1bfbedb0ba9971f86cc713e06bc09b62b72580bd99919f783"} Jun 09 00:18:07.059919 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.059868 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" podStartSLOduration=1.614504688 podStartE2EDuration="4.059852817s" totalImagesPullingTime="2.445348129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:07.059053568 +0000 UTC m=+450.283450837" watchObservedRunningTime="2026-06-09 00:18:07.059852817 +0000 UTC m=+450.284250084" Jun 09 00:18:07.073554 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.073510 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" podStartSLOduration=1.184628475 podStartE2EDuration="3.073496232s" totalImagesPullingTime="1.888867757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:07.072566196 +0000 UTC m=+450.296963466" watchObservedRunningTime="2026-06-09 00:18:07.073496232 +0000 UTC m=+450.297893500" Jun 09 00:18:07.297816 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.297792 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qmjfx_fea0a509-d31a-4035-95c1-496d0e018e5f/registry-server/0.log" Jun 09 00:18:07.299166 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.299150 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:07.365956 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.365924 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd7j8\" (UniqueName: \"kubernetes.io/projected/fea0a509-d31a-4035-95c1-496d0e018e5f-kube-api-access-fd7j8\") pod \"fea0a509-d31a-4035-95c1-496d0e018e5f\" (UID: \"fea0a509-d31a-4035-95c1-496d0e018e5f\") " Jun 09 00:18:07.368364 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.368331 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea0a509-d31a-4035-95c1-496d0e018e5f-kube-api-access-fd7j8" (OuterVolumeSpecName: "kube-api-access-fd7j8") pod "fea0a509-d31a-4035-95c1-496d0e018e5f" (UID: "fea0a509-d31a-4035-95c1-496d0e018e5f"). InnerVolumeSpecName "kube-api-access-fd7j8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:18:07.467319 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:07.467287 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd7j8\" (UniqueName: \"kubernetes.io/projected/fea0a509-d31a-4035-95c1-496d0e018e5f-kube-api-access-fd7j8\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:18:08.052281 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.052253 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qmjfx_fea0a509-d31a-4035-95c1-496d0e018e5f/registry-server/0.log" Jun 09 00:18:08.052667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.052294 2550 generic.go:356] "Generic (PLEG): container finished" podID="fea0a509-d31a-4035-95c1-496d0e018e5f" containerID="171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538" exitCode=1 Jun 09 00:18:08.052667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.052366 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" Jun 09 00:18:08.052667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.052392 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" event={"ID":"fea0a509-d31a-4035-95c1-496d0e018e5f","Type":"ContainerDied","Data":"171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538"} Jun 09 00:18:08.052667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.052467 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qmjfx" event={"ID":"fea0a509-d31a-4035-95c1-496d0e018e5f","Type":"ContainerDied","Data":"2a8f87cc32803356a29ea1c919c5f786911855900d7d946e789ac905637f147c"} Jun 09 00:18:08.052667 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.052488 2550 scope.go:117] "RemoveContainer" containerID="171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538" Jun 09 00:18:08.062125 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.062108 2550 scope.go:117] "RemoveContainer" containerID="171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538" Jun 09 00:18:08.062386 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:08.062367 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538\": container with ID starting with 171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538 not found: ID does not exist" containerID="171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538" Jun 09 00:18:08.062453 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.062392 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538"} err="failed to get container status \"171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538\": rpc error: code = NotFound desc = could not find container \"171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538\": container with ID starting with 171dc67cceb4103aff7d4228ccc04ef5034fd3cd593a4b1905d74320bec34538 not found: ID does not exist" Jun 09 00:18:08.074048 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.074027 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qmjfx"] Jun 09 00:18:08.076893 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:08.076867 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qmjfx"] Jun 09 00:18:09.236033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:09.236001 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea0a509-d31a-4035-95c1-496d0e018e5f" path="/var/lib/kubelet/pods/fea0a509-d31a-4035-95c1-496d0e018e5f/volumes" Jun 09 00:18:10.279042 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:10.279006 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:18:10.279309 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:10.279116 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:18:10.279309 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:10.279167 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:18:11.072338 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:11.072295 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" event={"ID":"e467d419-13df-4ca3-a5c0-2c520ed5ac73","Type":"ContainerStarted","Data":"d7c5f7d9328f89735cdd3ae5a7e903c540800d1023cf8acc3e08a76eb4b08f4d"} Jun 09 00:18:11.092700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:11.092645 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" podStartSLOduration=2.034235914 podStartE2EDuration="20.092627687s" totalImagesPullingTime="18.058391773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:11.091411152 +0000 UTC m=+454.315808432" watchObservedRunningTime="2026-06-09 00:18:11.092627687 +0000 UTC m=+454.317024954" Jun 09 00:18:12.061770 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:12.061725 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:18:12.066381 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:12.066357 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:18:12.077217 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:12.077195 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:18:12.077915 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:12.077896 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc5pcjh" Jun 09 00:18:14.890216 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:14.890183 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:14.890679 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:14.890250 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:14.913163 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:14.913138 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:15.113724 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:15.113698 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-wj4g8" Jun 09 00:18:37.683950 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.683912 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92"] Jun 09 00:18:37.684375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.684323 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea0a509-d31a-4035-95c1-496d0e018e5f" containerName="registry-server" Jun 09 00:18:37.684375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.684337 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea0a509-d31a-4035-95c1-496d0e018e5f" containerName="registry-server" Jun 09 00:18:37.684457 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.684407 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea0a509-d31a-4035-95c1-496d0e018e5f" containerName="registry-server" Jun 09 00:18:37.691885 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.691863 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:37.695122 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.695095 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rvk2b\"" Jun 09 00:18:37.702819 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.702792 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92"] Jun 09 00:18:37.848778 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.848732 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1511b21-9216-44ba-8ddf-96d73907f87a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:37.848778 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.848779 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7rvp\" (UniqueName: \"kubernetes.io/projected/b1511b21-9216-44ba-8ddf-96d73907f87a-kube-api-access-q7rvp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:37.949737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.949631 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1511b21-9216-44ba-8ddf-96d73907f87a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:37.949737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.949674 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7rvp\" (UniqueName: \"kubernetes.io/projected/b1511b21-9216-44ba-8ddf-96d73907f87a-kube-api-access-q7rvp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:37.950129 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.950065 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1511b21-9216-44ba-8ddf-96d73907f87a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:37.962837 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:37.962815 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7rvp\" (UniqueName: \"kubernetes.io/projected/b1511b21-9216-44ba-8ddf-96d73907f87a-kube-api-access-q7rvp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:38.004684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:38.004643 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:38.145400 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:38.145368 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92"] Jun 09 00:18:38.148459 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:38.148427 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1511b21_9216_44ba_8ddf_96d73907f87a.slice/crio-000852b63244f7c0b9ca293cd18f0d4b24d4a2853717d0c5a7df8e322a2221d4 WatchSource:0}: Error finding container 000852b63244f7c0b9ca293cd18f0d4b24d4a2853717d0c5a7df8e322a2221d4: Status 404 returned error can't find the container with id 000852b63244f7c0b9ca293cd18f0d4b24d4a2853717d0c5a7df8e322a2221d4 Jun 09 00:18:38.196394 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:38.196360 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" event={"ID":"b1511b21-9216-44ba-8ddf-96d73907f87a","Type":"ContainerStarted","Data":"000852b63244f7c0b9ca293cd18f0d4b24d4a2853717d0c5a7df8e322a2221d4"} Jun 09 00:18:40.013168 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.013130 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-947cs"] Jun 09 00:18:40.016880 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.016861 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:40.019246 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.019228 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-87mhn\"" Jun 09 00:18:40.026336 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.026315 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-947cs"] Jun 09 00:18:40.170095 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.170044 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb7kr\" (UniqueName: \"kubernetes.io/projected/47f50faf-df9b-48c6-a748-23daa9f15e76-kube-api-access-pb7kr\") pod \"authorino-operator-75cc8d5cb7-947cs\" (UID: \"47f50faf-df9b-48c6-a748-23daa9f15e76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:40.271328 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.271238 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb7kr\" (UniqueName: \"kubernetes.io/projected/47f50faf-df9b-48c6-a748-23daa9f15e76-kube-api-access-pb7kr\") pod \"authorino-operator-75cc8d5cb7-947cs\" (UID: \"47f50faf-df9b-48c6-a748-23daa9f15e76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:40.281876 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.281841 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb7kr\" (UniqueName: \"kubernetes.io/projected/47f50faf-df9b-48c6-a748-23daa9f15e76-kube-api-access-pb7kr\") pod \"authorino-operator-75cc8d5cb7-947cs\" (UID: \"47f50faf-df9b-48c6-a748-23daa9f15e76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:40.329361 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.329322 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:40.474396 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:40.474352 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-947cs"] Jun 09 00:18:40.477701 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:40.477667 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f50faf_df9b_48c6_a748_23daa9f15e76.slice/crio-736b0247b53a153b5d9feec2d9711ca33800839e7774cbdbf71e97e30a0ec833 WatchSource:0}: Error finding container 736b0247b53a153b5d9feec2d9711ca33800839e7774cbdbf71e97e30a0ec833: Status 404 returned error can't find the container with id 736b0247b53a153b5d9feec2d9711ca33800839e7774cbdbf71e97e30a0ec833 Jun 09 00:18:41.213465 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:41.213428 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" event={"ID":"47f50faf-df9b-48c6-a748-23daa9f15e76","Type":"ContainerStarted","Data":"736b0247b53a153b5d9feec2d9711ca33800839e7774cbdbf71e97e30a0ec833"} Jun 09 00:18:44.229790 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:44.229751 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" event={"ID":"47f50faf-df9b-48c6-a748-23daa9f15e76","Type":"ContainerStarted","Data":"aa0606f42c8c2aa0131baf1b01ae4ff7d66d6cf4ef8738fa2a9621b7fcc3931f"} Jun 09 00:18:44.230265 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:44.229815 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:44.251262 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:44.251208 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" podStartSLOduration=2.522618388 podStartE2EDuration="5.251193653s" totalImagesPullingTime="2.728575265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:44.248724057 +0000 UTC m=+487.473121328" watchObservedRunningTime="2026-06-09 00:18:44.251193653 +0000 UTC m=+487.475590921" Jun 09 00:18:45.436764 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.436726 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb"] Jun 09 00:18:45.441278 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.441229 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:18:45.445231 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.445204 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c6hm6\"" Jun 09 00:18:45.445370 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.445213 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 09 00:18:45.455776 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.455754 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb"] Jun 09 00:18:45.524613 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.524574 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d6gn\" (UniqueName: \"kubernetes.io/projected/26293972-949d-4dd0-b511-8da4ab08d7b1-kube-api-access-8d6gn\") pod \"dns-operator-controller-manager-864dd78c4c-jgwmb\" (UID: \"26293972-949d-4dd0-b511-8da4ab08d7b1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:18:45.625451 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.625412 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d6gn\" (UniqueName: \"kubernetes.io/projected/26293972-949d-4dd0-b511-8da4ab08d7b1-kube-api-access-8d6gn\") pod \"dns-operator-controller-manager-864dd78c4c-jgwmb\" (UID: \"26293972-949d-4dd0-b511-8da4ab08d7b1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:18:45.641300 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.641272 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d6gn\" (UniqueName: \"kubernetes.io/projected/26293972-949d-4dd0-b511-8da4ab08d7b1-kube-api-access-8d6gn\") pod \"dns-operator-controller-manager-864dd78c4c-jgwmb\" (UID: \"26293972-949d-4dd0-b511-8da4ab08d7b1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:18:45.756032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:45.755997 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:18:46.034762 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:46.034737 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb"] Jun 09 00:18:46.037768 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:46.037738 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26293972_949d_4dd0_b511_8da4ab08d7b1.slice/crio-2a748f77d174de1d46105fdb345e94f5d57ae0184e1f1de4ce69a89cdf35938c WatchSource:0}: Error finding container 2a748f77d174de1d46105fdb345e94f5d57ae0184e1f1de4ce69a89cdf35938c: Status 404 returned error can't find the container with id 2a748f77d174de1d46105fdb345e94f5d57ae0184e1f1de4ce69a89cdf35938c Jun 09 00:18:46.240504 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:46.240469 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" event={"ID":"26293972-949d-4dd0-b511-8da4ab08d7b1","Type":"ContainerStarted","Data":"2a748f77d174de1d46105fdb345e94f5d57ae0184e1f1de4ce69a89cdf35938c"} Jun 09 00:18:46.242066 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:46.242039 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" event={"ID":"b1511b21-9216-44ba-8ddf-96d73907f87a","Type":"ContainerStarted","Data":"3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39"} Jun 09 00:18:46.242211 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:46.242188 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:46.265652 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:46.265599 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" podStartSLOduration=1.482965889 podStartE2EDuration="9.265581179s" totalImagesPullingTime="7.78261529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:46.264492728 +0000 UTC m=+489.488889996" watchObservedRunningTime="2026-06-09 00:18:46.265581179 +0000 UTC m=+489.489978446" Jun 09 00:18:48.912429 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.912395 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c"] Jun 09 00:18:48.917491 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.917461 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:48.920954 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.920918 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 09 00:18:48.921124 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.920950 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 09 00:18:48.921355 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.921327 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ck9xr\"" Jun 09 00:18:48.926684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.926647 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c"] Jun 09 00:18:48.957141 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.957104 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6cd5181a-d2e2-4f9e-885b-e3df8df51424-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:48.957329 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.957237 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd5181a-d2e2-4f9e-885b-e3df8df51424-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:48.957329 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:48.957272 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm8zn\" (UniqueName: \"kubernetes.io/projected/6cd5181a-d2e2-4f9e-885b-e3df8df51424-kube-api-access-mm8zn\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.057850 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.057818 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6cd5181a-d2e2-4f9e-885b-e3df8df51424-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.057990 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.057940 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd5181a-d2e2-4f9e-885b-e3df8df51424-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.057990 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.057970 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm8zn\" (UniqueName: \"kubernetes.io/projected/6cd5181a-d2e2-4f9e-885b-e3df8df51424-kube-api-access-mm8zn\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.058864 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:49.058429 2550 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 09 00:18:49.058864 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:49.058495 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd5181a-d2e2-4f9e-885b-e3df8df51424-plugin-serving-cert podName:6cd5181a-d2e2-4f9e-885b-e3df8df51424 nodeName:}" failed. No retries permitted until 2026-06-09 00:18:49.558474825 +0000 UTC m=+492.782872072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6cd5181a-d2e2-4f9e-885b-e3df8df51424-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-h9z5c" (UID: "6cd5181a-d2e2-4f9e-885b-e3df8df51424") : secret "plugin-serving-cert" not found Jun 09 00:18:49.058864 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.058808 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6cd5181a-d2e2-4f9e-885b-e3df8df51424-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.066563 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.066542 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm8zn\" (UniqueName: \"kubernetes.io/projected/6cd5181a-d2e2-4f9e-885b-e3df8df51424-kube-api-access-mm8zn\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.259053 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.259010 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" event={"ID":"26293972-949d-4dd0-b511-8da4ab08d7b1","Type":"ContainerStarted","Data":"8cb6a673f1e30d59eae988eef8cde6918048dd43eef5a919ff120a2d88ce6bd6"} Jun 09 00:18:49.259291 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.259137 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:18:49.278375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.278320 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" podStartSLOduration=1.279835218 podStartE2EDuration="4.278305174s" totalImagesPullingTime="2.998469956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:49.276147696 +0000 UTC m=+492.500544980" watchObservedRunningTime="2026-06-09 00:18:49.278305174 +0000 UTC m=+492.502702441" Jun 09 00:18:49.462825 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.462790 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8694df9b48-4pvx8"] Jun 09 00:18:49.466437 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.466411 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.468929 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.468904 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 09 00:18:49.468929 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.468923 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 09 00:18:49.469157 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.468935 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dhtkt\"" Jun 09 00:18:49.469157 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.468914 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 09 00:18:49.469300 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.469280 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 09 00:18:49.469418 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.469279 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 09 00:18:49.469418 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.469323 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 09 00:18:49.469418 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.469327 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 09 00:18:49.474180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.474003 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 09 00:18:49.479050 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.479022 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8694df9b48-4pvx8"] Jun 09 00:18:49.561955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.561848 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqsc5\" (UniqueName: \"kubernetes.io/projected/0d8e401f-240a-49c2-b3ef-3b96b838071f-kube-api-access-hqsc5\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.561955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.561896 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-serving-cert\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.561955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.561915 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-oauth-serving-cert\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.562280 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.562052 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-service-ca\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.562280 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.562174 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-config\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.562280 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.562217 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-trusted-ca-bundle\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.562280 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.562268 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd5181a-d2e2-4f9e-885b-e3df8df51424-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.562426 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.562311 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-oauth-config\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.564872 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.564843 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd5181a-d2e2-4f9e-885b-e3df8df51424-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h9z5c\" (UID: \"6cd5181a-d2e2-4f9e-885b-e3df8df51424\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.663493 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663456 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-config\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.663493 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663494 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-trusted-ca-bundle\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.663777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663517 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-oauth-config\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.663777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663541 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqsc5\" (UniqueName: \"kubernetes.io/projected/0d8e401f-240a-49c2-b3ef-3b96b838071f-kube-api-access-hqsc5\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.663777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663570 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-serving-cert\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.663777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663597 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-oauth-serving-cert\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.663777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.663678 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-service-ca\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.664404 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.664378 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-config\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.664511 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.664489 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-oauth-serving-cert\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.664560 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.664489 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-service-ca\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.664771 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.664748 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d8e401f-240a-49c2-b3ef-3b96b838071f-trusted-ca-bundle\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.666395 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.666372 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-oauth-config\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.666483 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.666380 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d8e401f-240a-49c2-b3ef-3b96b838071f-console-serving-cert\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.673043 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.673021 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqsc5\" (UniqueName: \"kubernetes.io/projected/0d8e401f-240a-49c2-b3ef-3b96b838071f-kube-api-access-hqsc5\") pod \"console-8694df9b48-4pvx8\" (UID: \"0d8e401f-240a-49c2-b3ef-3b96b838071f\") " pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.777347 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.777307 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:49.836714 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.836677 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" Jun 09 00:18:49.927103 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.927060 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8694df9b48-4pvx8"] Jun 09 00:18:49.930130 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:49.930103 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d8e401f_240a_49c2_b3ef_3b96b838071f.slice/crio-a36ba209888aae9d063731a1d4d3850b37663eb50c1c9460cfa6dfc2d0ce1d11 WatchSource:0}: Error finding container a36ba209888aae9d063731a1d4d3850b37663eb50c1c9460cfa6dfc2d0ce1d11: Status 404 returned error can't find the container with id a36ba209888aae9d063731a1d4d3850b37663eb50c1c9460cfa6dfc2d0ce1d11 Jun 09 00:18:49.988286 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:49.988253 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c"] Jun 09 00:18:49.991675 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:49.991643 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cd5181a_d2e2_4f9e_885b_e3df8df51424.slice/crio-0a042059c5f033d2efbf9194815fda76ac006bec4935a25cfd07f19ec8fba775 WatchSource:0}: Error finding container 0a042059c5f033d2efbf9194815fda76ac006bec4935a25cfd07f19ec8fba775: Status 404 returned error can't find the container with id 0a042059c5f033d2efbf9194815fda76ac006bec4935a25cfd07f19ec8fba775 Jun 09 00:18:50.264936 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:50.264899 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" event={"ID":"6cd5181a-d2e2-4f9e-885b-e3df8df51424","Type":"ContainerStarted","Data":"0a042059c5f033d2efbf9194815fda76ac006bec4935a25cfd07f19ec8fba775"} Jun 09 00:18:50.266522 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:50.266497 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8694df9b48-4pvx8" event={"ID":"0d8e401f-240a-49c2-b3ef-3b96b838071f","Type":"ContainerStarted","Data":"96ba73bf8ec988037a583a6df1226baa8f46779df14c2ef2891fa8df194c5397"} Jun 09 00:18:50.266653 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:50.266542 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8694df9b48-4pvx8" event={"ID":"0d8e401f-240a-49c2-b3ef-3b96b838071f","Type":"ContainerStarted","Data":"a36ba209888aae9d063731a1d4d3850b37663eb50c1c9460cfa6dfc2d0ce1d11"} Jun 09 00:18:50.283324 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:50.283274 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8694df9b48-4pvx8" podStartSLOduration=1.283255389 podStartE2EDuration="1.283255389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:18:50.282069928 +0000 UTC m=+493.506467196" watchObservedRunningTime="2026-06-09 00:18:50.283255389 +0000 UTC m=+493.507652656" Jun 09 00:18:55.237672 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:55.237640 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-947cs" Jun 09 00:18:57.251213 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:57.251184 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:58.512467 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.512428 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx"] Jun 09 00:18:58.517276 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.517249 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:58.534385 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.534352 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx"] Jun 09 00:18:58.643663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.643618 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx"] Jun 09 00:18:58.644033 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:58.644010 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-rmzv7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" Jun 09 00:18:58.657727 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.657683 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmzv7\" (UniqueName: \"kubernetes.io/projected/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-kube-api-access-rmzv7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:58.657928 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.657749 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:58.758918 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.758877 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmzv7\" (UniqueName: \"kubernetes.io/projected/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-kube-api-access-rmzv7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:58.759140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.758938 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:58.759398 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.759376 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:58.772997 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:58.772927 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmzv7\" (UniqueName: \"kubernetes.io/projected/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-kube-api-access-rmzv7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:59.008364 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.008322 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx"] Jun 09 00:18:59.018820 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.018783 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx"] Jun 09 00:18:59.035646 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.035566 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92"] Jun 09 00:18:59.035803 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.035774 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" podUID="b1511b21-9216-44ba-8ddf-96d73907f87a" containerName="manager" containerID="cri-o://3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39" gracePeriod=2 Jun 09 00:18:59.043263 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.043131 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92"] Jun 09 00:18:59.067987 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.067955 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568"] Jun 09 00:18:59.068572 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.068547 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1511b21-9216-44ba-8ddf-96d73907f87a" containerName="manager" Jun 09 00:18:59.068572 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.068573 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1511b21-9216-44ba-8ddf-96d73907f87a" containerName="manager" Jun 09 00:18:59.068732 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.068649 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1511b21-9216-44ba-8ddf-96d73907f87a" containerName="manager" Jun 09 00:18:59.072746 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.072006 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.092491 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:59.092323 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7cn92\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="b1511b21-9216-44ba-8ddf-96d73907f87a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:59.099750 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.098250 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568"] Jun 09 00:18:59.163178 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.163060 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w568\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.163178 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.163179 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5mpz\" (UniqueName: \"kubernetes.io/projected/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-kube-api-access-p5mpz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w568\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.264036 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.263992 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5mpz\" (UniqueName: \"kubernetes.io/projected/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-kube-api-access-p5mpz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w568\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.264242 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.264113 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w568\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.264553 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.264533 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w568\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.273415 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.273352 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5mpz\" (UniqueName: \"kubernetes.io/projected/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-kube-api-access-p5mpz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w568\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.304561 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.304423 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:59.322800 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.322760 2550 generic.go:356] "Generic (PLEG): container finished" podID="b1511b21-9216-44ba-8ddf-96d73907f87a" containerID="3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39" exitCode=0 Jun 09 00:18:59.322988 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.322835 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7cn92" Jun 09 00:18:59.322988 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.322861 2550 scope.go:117] "RemoveContainer" containerID="3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39" Jun 09 00:18:59.322988 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.322973 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:59.331253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.331216 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:59.333404 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:59.333372 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:59.333720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.333697 2550 scope.go:117] "RemoveContainer" containerID="3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39" Jun 09 00:18:59.334039 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:59.334018 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39\": container with ID starting with 3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39 not found: ID does not exist" containerID="3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39" Jun 09 00:18:59.334195 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.334047 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39"} err="failed to get container status \"3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39\": rpc error: code = NotFound desc = could not find container \"3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39\": container with ID starting with 3bacc4fdb20667f723c240280036dbc36ed106512ad21d7615a13a4797a01e39 not found: ID does not exist" Jun 09 00:18:59.364571 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.364537 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7rvp\" (UniqueName: \"kubernetes.io/projected/b1511b21-9216-44ba-8ddf-96d73907f87a-kube-api-access-q7rvp\") pod \"b1511b21-9216-44ba-8ddf-96d73907f87a\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " Jun 09 00:18:59.364797 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.364592 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1511b21-9216-44ba-8ddf-96d73907f87a-extensions-socket-volume\") pod \"b1511b21-9216-44ba-8ddf-96d73907f87a\" (UID: \"b1511b21-9216-44ba-8ddf-96d73907f87a\") " Jun 09 00:18:59.365419 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.365227 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1511b21-9216-44ba-8ddf-96d73907f87a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b1511b21-9216-44ba-8ddf-96d73907f87a" (UID: "b1511b21-9216-44ba-8ddf-96d73907f87a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:18:59.367244 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.367219 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1511b21-9216-44ba-8ddf-96d73907f87a-kube-api-access-q7rvp" (OuterVolumeSpecName: "kube-api-access-q7rvp") pod "b1511b21-9216-44ba-8ddf-96d73907f87a" (UID: "b1511b21-9216-44ba-8ddf-96d73907f87a"). InnerVolumeSpecName "kube-api-access-q7rvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:18:59.418691 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.418654 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:18:59.466183 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.466150 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmzv7\" (UniqueName: \"kubernetes.io/projected/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-kube-api-access-rmzv7\") pod \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " Jun 09 00:18:59.466357 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.466248 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-extensions-socket-volume\") pod \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\" (UID: \"5b6887ae-16a6-4fd6-8ab6-acf5073bafa9\") " Jun 09 00:18:59.466563 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.466533 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" (UID: "5b6887ae-16a6-4fd6-8ab6-acf5073bafa9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:18:59.466666 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.466589 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7rvp\" (UniqueName: \"kubernetes.io/projected/b1511b21-9216-44ba-8ddf-96d73907f87a-kube-api-access-q7rvp\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:18:59.466666 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.466611 2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1511b21-9216-44ba-8ddf-96d73907f87a-extensions-socket-volume\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:18:59.469033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.469007 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-kube-api-access-rmzv7" (OuterVolumeSpecName: "kube-api-access-rmzv7") pod "5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" (UID: "5b6887ae-16a6-4fd6-8ab6-acf5073bafa9"). InnerVolumeSpecName "kube-api-access-rmzv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:18:59.568175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.568095 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmzv7\" (UniqueName: \"kubernetes.io/projected/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-kube-api-access-rmzv7\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:18:59.568175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.568129 2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9-extensions-socket-volume\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:18:59.580664 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.580633 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568"] Jun 09 00:18:59.584239 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:18:59.584210 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03aeeaff_dcc5_4828_a8a5_f893d6d9db43.slice/crio-5788a2ac8343265e64ebe55425bc265a895749feaef8066776ae6ac5bd5658c2 WatchSource:0}: Error finding container 5788a2ac8343265e64ebe55425bc265a895749feaef8066776ae6ac5bd5658c2: Status 404 returned error can't find the container with id 5788a2ac8343265e64ebe55425bc265a895749feaef8066776ae6ac5bd5658c2 Jun 09 00:18:59.652254 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:59.652220 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:18:59.778238 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.778197 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:59.778389 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.778266 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:59.784916 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:18:59.784890 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:18:59.787159 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:18:59.787121 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:19:00.269741 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.269710 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jgwmb" Jun 09 00:19:00.271977 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:19:00.271934 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:19:00.342172 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.342119 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" event={"ID":"03aeeaff-dcc5-4828-a8a5-f893d6d9db43","Type":"ContainerStarted","Data":"cbc592430530c0cf402c51b0c392876006f31783853f006f9562c4bf8f4c5ab0"} Jun 09 00:19:00.342358 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.342140 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:19:00.342358 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.342246 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:19:00.342358 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.342288 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" event={"ID":"03aeeaff-dcc5-4828-a8a5-f893d6d9db43","Type":"ContainerStarted","Data":"5788a2ac8343265e64ebe55425bc265a895749feaef8066776ae6ac5bd5658c2"} Jun 09 00:19:00.347754 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.347725 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8694df9b48-4pvx8" Jun 09 00:19:00.365582 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:19:00.365530 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:19:00.366015 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:00.365972 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" podStartSLOduration=1.365955471 podStartE2EDuration="1.365955471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:19:00.363623553 +0000 UTC m=+503.588020931" watchObservedRunningTime="2026-06-09 00:19:00.365955471 +0000 UTC m=+503.590352741" Jun 09 00:19:00.367253 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:19:00.367224 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-bnzjx\" is forbidden: User \"system:node:ip-10-0-143-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-86.ec2.internal' and this object" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bnzjx" Jun 09 00:19:01.238221 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:01.238183 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b6887ae-16a6-4fd6-8ab6-acf5073bafa9" path="/var/lib/kubelet/pods/5b6887ae-16a6-4fd6-8ab6-acf5073bafa9/volumes" Jun 09 00:19:01.238670 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:01.238590 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1511b21-9216-44ba-8ddf-96d73907f87a" path="/var/lib/kubelet/pods/b1511b21-9216-44ba-8ddf-96d73907f87a/volumes" Jun 09 00:19:11.350579 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:11.350546 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:19:16.289686 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.289643 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568"] Jun 09 00:19:16.290112 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.289843 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" podUID="03aeeaff-dcc5-4828-a8a5-f893d6d9db43" containerName="manager" containerID="cri-o://cbc592430530c0cf402c51b0c392876006f31783853f006f9562c4bf8f4c5ab0" gracePeriod=10 Jun 09 00:19:16.437621 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.437587 2550 generic.go:356] "Generic (PLEG): container finished" podID="03aeeaff-dcc5-4828-a8a5-f893d6d9db43" containerID="cbc592430530c0cf402c51b0c392876006f31783853f006f9562c4bf8f4c5ab0" exitCode=0 Jun 09 00:19:16.437850 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.437655 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" event={"ID":"03aeeaff-dcc5-4828-a8a5-f893d6d9db43","Type":"ContainerDied","Data":"cbc592430530c0cf402c51b0c392876006f31783853f006f9562c4bf8f4c5ab0"} Jun 09 00:19:16.439521 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.439492 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" event={"ID":"6cd5181a-d2e2-4f9e-885b-e3df8df51424","Type":"ContainerStarted","Data":"2a3becfb9281f248458359790bbeb5b25913bcfbd65fed36c174322b8fa2c635"} Jun 09 00:19:16.454756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.454705 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h9z5c" podStartSLOduration=2.723236881 podStartE2EDuration="28.454687283s" totalImagesPullingTime="25.731450402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:19:16.453611642 +0000 UTC m=+519.678008910" watchObservedRunningTime="2026-06-09 00:19:16.454687283 +0000 UTC m=+519.679084550" Jun 09 00:19:16.530502 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.530475 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2"] Jun 09 00:19:16.533189 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.533167 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.538375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.538350 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:19:16.547932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.547903 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2"] Jun 09 00:19:16.641034 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.640998 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5mpz\" (UniqueName: \"kubernetes.io/projected/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-kube-api-access-p5mpz\") pod \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " Jun 09 00:19:16.641253 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.641055 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-extensions-socket-volume\") pod \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\" (UID: \"03aeeaff-dcc5-4828-a8a5-f893d6d9db43\") " Jun 09 00:19:16.641333 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.641269 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfll2\" (UniqueName: \"kubernetes.io/projected/a57b8430-8248-42fe-a454-3dcf41a832e3-kube-api-access-hfll2\") pod \"kuadrant-operator-controller-manager-86878c769f-jfpm2\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.641333 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.641296 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a57b8430-8248-42fe-a454-3dcf41a832e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jfpm2\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.641519 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.641489 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03aeeaff-dcc5-4828-a8a5-f893d6d9db43" (UID: "03aeeaff-dcc5-4828-a8a5-f893d6d9db43"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:19:16.643377 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.643358 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-kube-api-access-p5mpz" (OuterVolumeSpecName: "kube-api-access-p5mpz") pod "03aeeaff-dcc5-4828-a8a5-f893d6d9db43" (UID: "03aeeaff-dcc5-4828-a8a5-f893d6d9db43"). InnerVolumeSpecName "kube-api-access-p5mpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:19:16.742158 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.742129 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfll2\" (UniqueName: \"kubernetes.io/projected/a57b8430-8248-42fe-a454-3dcf41a832e3-kube-api-access-hfll2\") pod \"kuadrant-operator-controller-manager-86878c769f-jfpm2\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.742269 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.742173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a57b8430-8248-42fe-a454-3dcf41a832e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jfpm2\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.742269 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.742206 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5mpz\" (UniqueName: \"kubernetes.io/projected/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-kube-api-access-p5mpz\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:19:16.742269 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.742216 2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03aeeaff-dcc5-4828-a8a5-f893d6d9db43-extensions-socket-volume\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:19:16.742550 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.742534 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a57b8430-8248-42fe-a454-3dcf41a832e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jfpm2\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.750519 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.750485 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfll2\" (UniqueName: \"kubernetes.io/projected/a57b8430-8248-42fe-a454-3dcf41a832e3-kube-api-access-hfll2\") pod \"kuadrant-operator-controller-manager-86878c769f-jfpm2\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.846891 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.846799 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:16.985245 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:16.985217 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2"] Jun 09 00:19:16.988051 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:19:16.988020 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57b8430_8248_42fe_a454_3dcf41a832e3.slice/crio-9837c42512bf305d73c6d6eeb4589a949989a45f69392fb94b7a1274e056d5b1 WatchSource:0}: Error finding container 9837c42512bf305d73c6d6eeb4589a949989a45f69392fb94b7a1274e056d5b1: Status 404 returned error can't find the container with id 9837c42512bf305d73c6d6eeb4589a949989a45f69392fb94b7a1274e056d5b1 Jun 09 00:19:17.446191 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.446151 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" event={"ID":"a57b8430-8248-42fe-a454-3dcf41a832e3","Type":"ContainerStarted","Data":"dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5"} Jun 09 00:19:17.446629 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.446204 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" event={"ID":"a57b8430-8248-42fe-a454-3dcf41a832e3","Type":"ContainerStarted","Data":"9837c42512bf305d73c6d6eeb4589a949989a45f69392fb94b7a1274e056d5b1"} Jun 09 00:19:17.446850 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.446829 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:17.455006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.454978 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" Jun 09 00:19:17.455180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.454995 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568" event={"ID":"03aeeaff-dcc5-4828-a8a5-f893d6d9db43","Type":"ContainerDied","Data":"5788a2ac8343265e64ebe55425bc265a895749feaef8066776ae6ac5bd5658c2"} Jun 09 00:19:17.455180 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.455094 2550 scope.go:117] "RemoveContainer" containerID="cbc592430530c0cf402c51b0c392876006f31783853f006f9562c4bf8f4c5ab0" Jun 09 00:19:17.467056 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.466969 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" podStartSLOduration=1.46694983 podStartE2EDuration="1.46694983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:19:17.466613764 +0000 UTC m=+520.691011034" watchObservedRunningTime="2026-06-09 00:19:17.46694983 +0000 UTC m=+520.691347099" Jun 09 00:19:17.481877 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.481845 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568"] Jun 09 00:19:17.485542 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:17.485512 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w568"] Jun 09 00:19:19.236142 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:19.236110 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03aeeaff-dcc5-4828-a8a5-f893d6d9db43" path="/var/lib/kubelet/pods/03aeeaff-dcc5-4828-a8a5-f893d6d9db43/volumes" Jun 09 00:19:28.463606 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:28.463578 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:19:32.569750 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.569714 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59"] Jun 09 00:19:32.570921 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.570447 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03aeeaff-dcc5-4828-a8a5-f893d6d9db43" containerName="manager" Jun 09 00:19:32.570921 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.570476 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="03aeeaff-dcc5-4828-a8a5-f893d6d9db43" containerName="manager" Jun 09 00:19:32.570921 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.570579 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="03aeeaff-dcc5-4828-a8a5-f893d6d9db43" containerName="manager" Jun 09 00:19:32.664700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.664661 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59"] Jun 09 00:19:32.664892 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.664780 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.667423 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.667400 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bcgfl\"" Jun 09 00:19:32.788949 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.788913 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx4q5\" (UniqueName: \"kubernetes.io/projected/739ee8d3-2160-452a-be22-14f55d145173-kube-api-access-zx4q5\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.788949 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.788954 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739ee8d3-2160-452a-be22-14f55d145173-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.788991 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739ee8d3-2160-452a-be22-14f55d145173-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.789056 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.789119 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.789140 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739ee8d3-2160-452a-be22-14f55d145173-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.789157 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789453 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.789248 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.789453 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.789286 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890361 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739ee8d3-2160-452a-be22-14f55d145173-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890450 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890416 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739ee8d3-2160-452a-be22-14f55d145173-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890456 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890487 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890513 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739ee8d3-2160-452a-be22-14f55d145173-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890541 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890600 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890684 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890655 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890718 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx4q5\" (UniqueName: \"kubernetes.io/projected/739ee8d3-2160-452a-be22-14f55d145173-kube-api-access-zx4q5\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.890981 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.890955 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.891091 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.891011 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.891186 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.891169 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.891267 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.891246 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.891432 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.891415 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739ee8d3-2160-452a-be22-14f55d145173-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.893514 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.893485 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739ee8d3-2160-452a-be22-14f55d145173-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.893642 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.893624 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739ee8d3-2160-452a-be22-14f55d145173-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.897875 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.897853 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739ee8d3-2160-452a-be22-14f55d145173-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.897986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.897911 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx4q5\" (UniqueName: \"kubernetes.io/projected/739ee8d3-2160-452a-be22-14f55d145173-kube-api-access-zx4q5\") pod \"maas-default-gateway-openshift-default-8559cd5744-zwn59\" (UID: \"739ee8d3-2160-452a-be22-14f55d145173\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:32.975627 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:32.975594 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:33.142015 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.141991 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59"] Jun 09 00:19:33.143679 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:19:33.143645 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod739ee8d3_2160_452a_be22_14f55d145173.slice/crio-7970b8ec916897e4060030a0237e9cb8e59eeb1e0aae8994548db92f8ac4e64a WatchSource:0}: Error finding container 7970b8ec916897e4060030a0237e9cb8e59eeb1e0aae8994548db92f8ac4e64a: Status 404 returned error can't find the container with id 7970b8ec916897e4060030a0237e9cb8e59eeb1e0aae8994548db92f8ac4e64a Jun 09 00:19:33.146005 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.145971 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:19:33.146105 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.146041 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:19:33.146105 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.146069 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:19:33.536889 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.536857 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" event={"ID":"739ee8d3-2160-452a-be22-14f55d145173","Type":"ContainerStarted","Data":"d87651a885b958b3d6a22b7731f1f6376998338fffe7cbabed8d333a90f1641d"} Jun 09 00:19:33.537053 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.536908 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" event={"ID":"739ee8d3-2160-452a-be22-14f55d145173","Type":"ContainerStarted","Data":"7970b8ec916897e4060030a0237e9cb8e59eeb1e0aae8994548db92f8ac4e64a"} Jun 09 00:19:33.557710 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.557654 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" podStartSLOduration=1.557639032 podStartE2EDuration="1.557639032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:19:33.556382644 +0000 UTC m=+536.780779914" watchObservedRunningTime="2026-06-09 00:19:33.557639032 +0000 UTC m=+536.782036301" Jun 09 00:19:33.975961 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:33.975925 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:34.980972 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:34.980944 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:35.547458 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:35.547424 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:19:35.548517 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:19:35.548492 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:20:49.069690 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.069592 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:20:49.073348 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.073324 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:49.075632 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.075609 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wdcp9\"" Jun 09 00:20:49.075759 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.075609 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 00:20:49.075759 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.075609 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 00:20:49.076521 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.076503 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 00:20:49.084042 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.084022 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:20:49.106663 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.106631 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8g5q\" (UniqueName: \"kubernetes.io/projected/6ae89e8f-6d16-4d28-a62b-3de6490d30d2-kube-api-access-b8g5q\") pod \"maas-keycloak-0\" (UID: \"6ae89e8f-6d16-4d28-a62b-3de6490d30d2\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:49.208023 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.207985 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8g5q\" (UniqueName: \"kubernetes.io/projected/6ae89e8f-6d16-4d28-a62b-3de6490d30d2-kube-api-access-b8g5q\") pod \"maas-keycloak-0\" (UID: \"6ae89e8f-6d16-4d28-a62b-3de6490d30d2\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:49.216231 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.216208 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8g5q\" (UniqueName: \"kubernetes.io/projected/6ae89e8f-6d16-4d28-a62b-3de6490d30d2-kube-api-access-b8g5q\") pod \"maas-keycloak-0\" (UID: \"6ae89e8f-6d16-4d28-a62b-3de6490d30d2\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:49.384693 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.384620 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:49.524298 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.524259 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:20:49.527312 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:20:49.527284 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae89e8f_6d16_4d28_a62b_3de6490d30d2.slice/crio-f542f9c8ef524224ce0485f1e36d19d8c504eacfdf35aadf6cc1b49399903fd6 WatchSource:0}: Error finding container f542f9c8ef524224ce0485f1e36d19d8c504eacfdf35aadf6cc1b49399903fd6: Status 404 returned error can't find the container with id f542f9c8ef524224ce0485f1e36d19d8c504eacfdf35aadf6cc1b49399903fd6 Jun 09 00:20:49.917019 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:49.916983 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6ae89e8f-6d16-4d28-a62b-3de6490d30d2","Type":"ContainerStarted","Data":"f542f9c8ef524224ce0485f1e36d19d8c504eacfdf35aadf6cc1b49399903fd6"} Jun 09 00:20:55.958842 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:55.958780 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6ae89e8f-6d16-4d28-a62b-3de6490d30d2","Type":"ContainerStarted","Data":"1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572"} Jun 09 00:20:55.978176 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:55.978118 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.307889838 podStartE2EDuration="6.978102072s" totalImagesPullingTime="5.670212234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:20:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:20:55.975986594 +0000 UTC m=+619.200383909" watchObservedRunningTime="2026-06-09 00:20:55.978102072 +0000 UTC m=+619.202499337" Jun 09 00:20:56.385322 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:56.385229 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:56.387059 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:56.387011 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:20:57.385927 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:57.385881 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:20:58.386038 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:58.385986 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:20:59.385009 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:59.384963 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 00:20:59.385383 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:20:59.385344 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:00.385200 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:00.385153 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:01.385620 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:01.385567 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:02.385960 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:02.385898 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:03.385948 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:03.385895 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:04.385955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:04.385889 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:05.385312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:05.385197 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:06.386148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:06.386063 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:07.385060 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:07.385014 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:08.385711 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:08.385658 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused" Jun 09 00:21:09.516631 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:09.516589 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 00:21:09.544347 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:09.544256 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 00:21:19.523853 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:19.523813 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 00:21:25.380117 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.380062 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8646c59745-np4j4"] Jun 09 00:21:25.384252 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.384220 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:25.386863 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.386835 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wql6k\"" Jun 09 00:21:25.387005 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.386906 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 00:21:25.387674 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.387652 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 00:21:25.402338 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.402315 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8646c59745-np4j4"] Jun 09 00:21:25.490017 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.489983 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trph2\" (UniqueName: \"kubernetes.io/projected/9d9cf854-3a43-4f49-a218-118f511fb1ca-kube-api-access-trph2\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:25.490228 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.490167 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:25.560861 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.560824 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58c6c59df9-b7t25"] Jun 09 00:21:25.563291 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.563275 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.574821 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.574508 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-b7t25"] Jun 09 00:21:25.591423 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.591387 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trph2\" (UniqueName: \"kubernetes.io/projected/9d9cf854-3a43-4f49-a218-118f511fb1ca-kube-api-access-trph2\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:25.591578 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.591534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:25.591675 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:21:25.591661 2550 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 00:21:25.591741 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:21:25.591730 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert podName:9d9cf854-3a43-4f49-a218-118f511fb1ca nodeName:}" failed. No retries permitted until 2026-06-09 00:21:26.091708266 +0000 UTC m=+649.316105529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert") pod "maas-controller-8646c59745-np4j4" (UID: "9d9cf854-3a43-4f49-a218-118f511fb1ca") : secret "maas-controller-webhook-cert" not found Jun 09 00:21:25.602349 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.602316 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trph2\" (UniqueName: \"kubernetes.io/projected/9d9cf854-3a43-4f49-a218-118f511fb1ca-kube-api-access-trph2\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:25.692591 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.692494 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8fdde4-0c17-445a-8cae-086377aad5d1-webhook-cert\") pod \"maas-controller-58c6c59df9-b7t25\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.692591 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.692552 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlsq7\" (UniqueName: \"kubernetes.io/projected/cc8fdde4-0c17-445a-8cae-086377aad5d1-kube-api-access-hlsq7\") pod \"maas-controller-58c6c59df9-b7t25\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.793370 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.793332 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq7\" (UniqueName: \"kubernetes.io/projected/cc8fdde4-0c17-445a-8cae-086377aad5d1-kube-api-access-hlsq7\") pod \"maas-controller-58c6c59df9-b7t25\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.793535 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.793461 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8fdde4-0c17-445a-8cae-086377aad5d1-webhook-cert\") pod \"maas-controller-58c6c59df9-b7t25\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.795977 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.795947 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8fdde4-0c17-445a-8cae-086377aad5d1-webhook-cert\") pod \"maas-controller-58c6c59df9-b7t25\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.801555 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.801535 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlsq7\" (UniqueName: \"kubernetes.io/projected/cc8fdde4-0c17-445a-8cae-086377aad5d1-kube-api-access-hlsq7\") pod \"maas-controller-58c6c59df9-b7t25\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:25.881260 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:25.881219 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:26.022393 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.022366 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-b7t25"] Jun 09 00:21:26.024639 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:21:26.024611 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8fdde4_0c17_445a_8cae_086377aad5d1.slice/crio-c07e0da7dd777d6e161c208021882dd9bd8189590383b4927b0391034f673fa0 WatchSource:0}: Error finding container c07e0da7dd777d6e161c208021882dd9bd8189590383b4927b0391034f673fa0: Status 404 returned error can't find the container with id c07e0da7dd777d6e161c208021882dd9bd8189590383b4927b0391034f673fa0 Jun 09 00:21:26.026214 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.026198 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 00:21:26.095877 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.095840 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:26.098387 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.098369 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert\") pod \"maas-controller-8646c59745-np4j4\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:26.135024 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.134985 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c6c59df9-b7t25" event={"ID":"cc8fdde4-0c17-445a-8cae-086377aad5d1","Type":"ContainerStarted","Data":"c07e0da7dd777d6e161c208021882dd9bd8189590383b4927b0391034f673fa0"} Jun 09 00:21:26.298409 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.298319 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:26.437962 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:26.437889 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8646c59745-np4j4"] Jun 09 00:21:26.440398 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:21:26.440371 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9cf854_3a43_4f49_a218_118f511fb1ca.slice/crio-9bebba0bde8ca9b9c53ff594ac6262f25df69eeffb64555b4d029c83bcd66829 WatchSource:0}: Error finding container 9bebba0bde8ca9b9c53ff594ac6262f25df69eeffb64555b4d029c83bcd66829: Status 404 returned error can't find the container with id 9bebba0bde8ca9b9c53ff594ac6262f25df69eeffb64555b4d029c83bcd66829 Jun 09 00:21:27.144149 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:27.144040 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8646c59745-np4j4" event={"ID":"9d9cf854-3a43-4f49-a218-118f511fb1ca","Type":"ContainerStarted","Data":"9bebba0bde8ca9b9c53ff594ac6262f25df69eeffb64555b4d029c83bcd66829"} Jun 09 00:21:29.156902 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:29.156815 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c6c59df9-b7t25" event={"ID":"cc8fdde4-0c17-445a-8cae-086377aad5d1","Type":"ContainerStarted","Data":"9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4"} Jun 09 00:21:29.157369 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:29.156943 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:29.158483 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:29.158457 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8646c59745-np4j4" event={"ID":"9d9cf854-3a43-4f49-a218-118f511fb1ca","Type":"ContainerStarted","Data":"de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0"} Jun 09 00:21:29.158607 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:29.158534 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:29.175723 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:29.175667 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58c6c59df9-b7t25" podStartSLOduration=1.469666625 podStartE2EDuration="4.175654797s" totalImagesPullingTime="2.705988172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:21:29.174003611 +0000 UTC m=+652.398400910" watchObservedRunningTime="2026-06-09 00:21:29.175654797 +0000 UTC m=+652.400052064" Jun 09 00:21:29.196618 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:29.196565 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8646c59745-np4j4" podStartSLOduration=1.895518713 podStartE2EDuration="4.196541616s" totalImagesPullingTime="2.301022903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:21:29.194221143 +0000 UTC m=+652.418618408" watchObservedRunningTime="2026-06-09 00:21:29.196541616 +0000 UTC m=+652.420938883" Jun 09 00:21:40.169871 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.169837 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:40.170312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.170013 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:21:40.229000 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.228957 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8646c59745-np4j4"] Jun 09 00:21:40.229284 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.229227 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-8646c59745-np4j4" podUID="9d9cf854-3a43-4f49-a218-118f511fb1ca" containerName="manager" containerID="cri-o://de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0" gracePeriod=30 Jun 09 00:21:40.485771 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.485748 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:40.641577 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.641550 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trph2\" (UniqueName: \"kubernetes.io/projected/9d9cf854-3a43-4f49-a218-118f511fb1ca-kube-api-access-trph2\") pod \"9d9cf854-3a43-4f49-a218-118f511fb1ca\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " Jun 09 00:21:40.641740 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.641690 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert\") pod \"9d9cf854-3a43-4f49-a218-118f511fb1ca\" (UID: \"9d9cf854-3a43-4f49-a218-118f511fb1ca\") " Jun 09 00:21:40.643796 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.643768 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9d9cf854-3a43-4f49-a218-118f511fb1ca" (UID: "9d9cf854-3a43-4f49-a218-118f511fb1ca"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:21:40.643895 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.643817 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d9cf854-3a43-4f49-a218-118f511fb1ca-kube-api-access-trph2" (OuterVolumeSpecName: "kube-api-access-trph2") pod "9d9cf854-3a43-4f49-a218-118f511fb1ca" (UID: "9d9cf854-3a43-4f49-a218-118f511fb1ca"). InnerVolumeSpecName "kube-api-access-trph2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:21:40.743004 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.742913 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d9cf854-3a43-4f49-a218-118f511fb1ca-webhook-cert\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:21:40.743004 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:40.742948 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trph2\" (UniqueName: \"kubernetes.io/projected/9d9cf854-3a43-4f49-a218-118f511fb1ca-kube-api-access-trph2\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:21:41.216824 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.216791 2550 generic.go:356] "Generic (PLEG): container finished" podID="9d9cf854-3a43-4f49-a218-118f511fb1ca" containerID="de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0" exitCode=0 Jun 09 00:21:41.217308 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.216839 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8646c59745-np4j4" Jun 09 00:21:41.217308 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.216841 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8646c59745-np4j4" event={"ID":"9d9cf854-3a43-4f49-a218-118f511fb1ca","Type":"ContainerDied","Data":"de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0"} Jun 09 00:21:41.217308 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.216898 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8646c59745-np4j4" event={"ID":"9d9cf854-3a43-4f49-a218-118f511fb1ca","Type":"ContainerDied","Data":"9bebba0bde8ca9b9c53ff594ac6262f25df69eeffb64555b4d029c83bcd66829"} Jun 09 00:21:41.217308 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.216921 2550 scope.go:117] "RemoveContainer" containerID="de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0" Jun 09 00:21:41.226804 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.226781 2550 scope.go:117] "RemoveContainer" containerID="de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0" Jun 09 00:21:41.227178 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:21:41.227067 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0\": container with ID starting with de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0 not found: ID does not exist" containerID="de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0" Jun 09 00:21:41.227262 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.227189 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0"} err="failed to get container status \"de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0\": rpc error: code = NotFound desc = could not find container \"de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0\": container with ID starting with de11b4cce0ff43705c50be6058fa686c8ad02634fe1a3fb797a6d6aea9cde0d0 not found: ID does not exist" Jun 09 00:21:41.240990 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.240961 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8646c59745-np4j4"] Jun 09 00:21:41.244491 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:41.244467 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8646c59745-np4j4"] Jun 09 00:21:43.236062 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:21:43.236028 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d9cf854-3a43-4f49-a218-118f511fb1ca" path="/var/lib/kubelet/pods/9d9cf854-3a43-4f49-a218-118f511fb1ca/volumes" Jun 09 00:22:00.111459 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.111421 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m"] Jun 09 00:22:00.112015 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.112000 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9cf854-3a43-4f49-a218-118f511fb1ca" containerName="manager" Jun 09 00:22:00.112062 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.112020 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9cf854-3a43-4f49-a218-118f511fb1ca" containerName="manager" Jun 09 00:22:00.112142 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.112131 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d9cf854-3a43-4f49-a218-118f511fb1ca" containerName="manager" Jun 09 00:22:00.115823 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.115798 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.118459 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.118435 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jun 09 00:22:00.118597 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.118446 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 09 00:22:00.118597 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.118475 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-dockercfg-nrr7v\"" Jun 09 00:22:00.118597 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.118473 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 09 00:22:00.123737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.123714 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m"] Jun 09 00:22:00.223993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.223963 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbz6r\" (UniqueName: \"kubernetes.io/projected/2df2c3b9-05c7-451f-8327-45aa97e3f710-kube-api-access-hbz6r\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.224172 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.224006 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2df2c3b9-05c7-451f-8327-45aa97e3f710-maas-api-tls\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.324662 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.324624 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbz6r\" (UniqueName: \"kubernetes.io/projected/2df2c3b9-05c7-451f-8327-45aa97e3f710-kube-api-access-hbz6r\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.324662 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.324667 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2df2c3b9-05c7-451f-8327-45aa97e3f710-maas-api-tls\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.324884 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:22:00.324769 2550 secret.go:189] Couldn't get secret redhat-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 09 00:22:00.324884 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:22:00.324822 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df2c3b9-05c7-451f-8327-45aa97e3f710-maas-api-tls podName:2df2c3b9-05c7-451f-8327-45aa97e3f710 nodeName:}" failed. No retries permitted until 2026-06-09 00:22:00.824805727 +0000 UTC m=+684.049202972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2df2c3b9-05c7-451f-8327-45aa97e3f710-maas-api-tls") pod "maas-api-ccb76f6bf-9g75m" (UID: "2df2c3b9-05c7-451f-8327-45aa97e3f710") : secret "maas-api-serving-cert" not found Jun 09 00:22:00.333666 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.333636 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbz6r\" (UniqueName: \"kubernetes.io/projected/2df2c3b9-05c7-451f-8327-45aa97e3f710-kube-api-access-hbz6r\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.568316 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.568268 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-5brzx"] Jun 09 00:22:00.571991 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.571973 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:00.574310 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.574285 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 09 00:22:00.574429 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.574368 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-djtps\"" Jun 09 00:22:00.583472 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.583446 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-5brzx"] Jun 09 00:22:00.624518 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.624484 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-q6lhg"] Jun 09 00:22:00.627698 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.627673 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jsxl\" (UniqueName: \"kubernetes.io/projected/b94d3fe5-f1d0-4ff4-9f5d-a973028d8303-kube-api-access-4jsxl\") pod \"payload-pre-processing-67d7689b65-5brzx\" (UID: \"b94d3fe5-f1d0-4ff4-9f5d-a973028d8303\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:00.627952 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.627937 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:00.637872 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.637842 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-q6lhg"] Jun 09 00:22:00.728398 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.728365 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jsxl\" (UniqueName: \"kubernetes.io/projected/b94d3fe5-f1d0-4ff4-9f5d-a973028d8303-kube-api-access-4jsxl\") pod \"payload-pre-processing-67d7689b65-5brzx\" (UID: \"b94d3fe5-f1d0-4ff4-9f5d-a973028d8303\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:00.728579 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.728442 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skkdb\" (UniqueName: \"kubernetes.io/projected/592a56c4-98b0-40cc-9ad7-d47627b8e6df-kube-api-access-skkdb\") pod \"payload-processing-7c6c859784-q6lhg\" (UID: \"592a56c4-98b0-40cc-9ad7-d47627b8e6df\") " pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:00.736638 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.736608 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jsxl\" (UniqueName: \"kubernetes.io/projected/b94d3fe5-f1d0-4ff4-9f5d-a973028d8303-kube-api-access-4jsxl\") pod \"payload-pre-processing-67d7689b65-5brzx\" (UID: \"b94d3fe5-f1d0-4ff4-9f5d-a973028d8303\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:00.830044 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.829947 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2df2c3b9-05c7-451f-8327-45aa97e3f710-maas-api-tls\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.830044 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.829986 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skkdb\" (UniqueName: \"kubernetes.io/projected/592a56c4-98b0-40cc-9ad7-d47627b8e6df-kube-api-access-skkdb\") pod \"payload-processing-7c6c859784-q6lhg\" (UID: \"592a56c4-98b0-40cc-9ad7-d47627b8e6df\") " pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:00.832648 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.832624 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2df2c3b9-05c7-451f-8327-45aa97e3f710-maas-api-tls\") pod \"maas-api-ccb76f6bf-9g75m\" (UID: \"2df2c3b9-05c7-451f-8327-45aa97e3f710\") " pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:00.838776 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.838750 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skkdb\" (UniqueName: \"kubernetes.io/projected/592a56c4-98b0-40cc-9ad7-d47627b8e6df-kube-api-access-skkdb\") pod \"payload-processing-7c6c859784-q6lhg\" (UID: \"592a56c4-98b0-40cc-9ad7-d47627b8e6df\") " pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:00.885585 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.885545 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:00.940410 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:00.940373 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:01.028169 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.028128 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:01.053390 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.053336 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-5brzx"] Jun 09 00:22:01.056118 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:22:01.056060 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94d3fe5_f1d0_4ff4_9f5d_a973028d8303.slice/crio-c97436fc941243ab9f028b861a93f6a79273992b96dfff2f15635c409551d847 WatchSource:0}: Error finding container c97436fc941243ab9f028b861a93f6a79273992b96dfff2f15635c409551d847: Status 404 returned error can't find the container with id c97436fc941243ab9f028b861a93f6a79273992b96dfff2f15635c409551d847 Jun 09 00:22:01.122428 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.122397 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-q6lhg"] Jun 09 00:22:01.125936 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:22:01.125901 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592a56c4_98b0_40cc_9ad7_d47627b8e6df.slice/crio-55bb14654df89021b7b0934db96da065a4c0332485ca1b3b9be3a8af5069c6fe WatchSource:0}: Error finding container 55bb14654df89021b7b0934db96da065a4c0332485ca1b3b9be3a8af5069c6fe: Status 404 returned error can't find the container with id 55bb14654df89021b7b0934db96da065a4c0332485ca1b3b9be3a8af5069c6fe Jun 09 00:22:01.228996 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.228970 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m"] Jun 09 00:22:01.231050 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:22:01.231024 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df2c3b9_05c7_451f_8327_45aa97e3f710.slice/crio-7e50b4d8bee9e44bcc5f42a48a02200eb51deb0920b0ac4a542d73b6f4ebf3b2 WatchSource:0}: Error finding container 7e50b4d8bee9e44bcc5f42a48a02200eb51deb0920b0ac4a542d73b6f4ebf3b2: Status 404 returned error can't find the container with id 7e50b4d8bee9e44bcc5f42a48a02200eb51deb0920b0ac4a542d73b6f4ebf3b2 Jun 09 00:22:01.316652 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.316611 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" event={"ID":"2df2c3b9-05c7-451f-8327-45aa97e3f710","Type":"ContainerStarted","Data":"7e50b4d8bee9e44bcc5f42a48a02200eb51deb0920b0ac4a542d73b6f4ebf3b2"} Jun 09 00:22:01.317872 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.317841 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" event={"ID":"b94d3fe5-f1d0-4ff4-9f5d-a973028d8303","Type":"ContainerStarted","Data":"c97436fc941243ab9f028b861a93f6a79273992b96dfff2f15635c409551d847"} Jun 09 00:22:01.318889 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:01.318868 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"55bb14654df89021b7b0934db96da065a4c0332485ca1b3b9be3a8af5069c6fe"} Jun 09 00:22:04.341435 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.341394 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" event={"ID":"2df2c3b9-05c7-451f-8327-45aa97e3f710","Type":"ContainerStarted","Data":"2355595f3c8bea39a9ada79d449d78466ba47420b4131a1ecd74ba7465c2eb49"} Jun 09 00:22:04.342133 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.341578 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:04.343011 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.342989 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" event={"ID":"b94d3fe5-f1d0-4ff4-9f5d-a973028d8303","Type":"ContainerStarted","Data":"5cfc3aaea8a56e6f527df04dc00eb3fd774af45fba2a232b763c68031383e277"} Jun 09 00:22:04.343158 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.343043 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:04.344491 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.344428 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"b5be139b45df3941ff0669b8c7834822063383818110718c41de073c2a01caff"} Jun 09 00:22:04.344589 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.344492 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:04.358207 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.358164 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" podStartSLOduration=2.351529806 podStartE2EDuration="4.358148816s" totalImagesPullingTime="2.00661901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:22:04.356149737 +0000 UTC m=+687.580547008" watchObservedRunningTime="2026-06-09 00:22:04.358148816 +0000 UTC m=+687.582546083" Jun 09 00:22:04.372955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.372886 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" podStartSLOduration=2.192215516 podStartE2EDuration="4.372866691s" totalImagesPullingTime="2.180651175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:22:04.369808056 +0000 UTC m=+687.594205326" watchObservedRunningTime="2026-06-09 00:22:04.372866691 +0000 UTC m=+687.597263960" Jun 09 00:22:04.387445 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:04.387379 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podStartSLOduration=2.274336322 podStartE2EDuration="4.387355629s" totalImagesPullingTime="2.113019307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:22:04.385320208 +0000 UTC m=+687.609717479" watchObservedRunningTime="2026-06-09 00:22:04.387355629 +0000 UTC m=+687.611752901" Jun 09 00:22:06.589284 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:06.589236 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zwn59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jun 09 00:22:06.589679 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:06.589306 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" podUID="739ee8d3-2160-452a-be22-14f55d145173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jun 09 00:22:08.376031 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:08.375927 2550 generic.go:356] "Generic (PLEG): container finished" podID="739ee8d3-2160-452a-be22-14f55d145173" containerID="d87651a885b958b3d6a22b7731f1f6376998338fffe7cbabed8d333a90f1641d" exitCode=0 Jun 09 00:22:08.376031 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:08.376014 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" event={"ID":"739ee8d3-2160-452a-be22-14f55d145173","Type":"ContainerDied","Data":"d87651a885b958b3d6a22b7731f1f6376998338fffe7cbabed8d333a90f1641d"} Jun 09 00:22:08.376532 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:08.376519 2550 scope.go:117] "RemoveContainer" containerID="d87651a885b958b3d6a22b7731f1f6376998338fffe7cbabed8d333a90f1641d" Jun 09 00:22:08.377572 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:08.377546 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:22:08.377628 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:08.377614 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:22:08.377664 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:08.377646 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 00:22:09.387587 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:09.387541 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" event={"ID":"739ee8d3-2160-452a-be22-14f55d145173","Type":"ContainerStarted","Data":"7be84ba6ee1e61f6c3804f14328cc5bc2c16f66c790366c8f8a55692d5b34a35"} Jun 09 00:22:09.976652 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:09.976561 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:22:09.976983 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:09.976958 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zwn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jun 09 00:22:09.977204 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:09.977176 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" podUID="739ee8d3-2160-452a-be22-14f55d145173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jun 09 00:22:10.356491 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:10.356405 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/maas-api-ccb76f6bf-9g75m" Jun 09 00:22:10.976913 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:10.976861 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zwn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jun 09 00:22:10.977440 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:10.976950 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" podUID="739ee8d3-2160-452a-be22-14f55d145173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jun 09 00:22:11.976197 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:11.976152 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zwn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jun 09 00:22:11.976493 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:11.976223 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" podUID="739ee8d3-2160-452a-be22-14f55d145173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jun 09 00:22:12.166347 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:12.166309 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:22:12.166788 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:12.166507 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" containerID="cri-o://1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572" gracePeriod=30 Jun 09 00:22:12.976262 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:12.976217 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zwn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jun 09 00:22:12.976488 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:12.976285 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" podUID="739ee8d3-2160-452a-be22-14f55d145173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jun 09 00:22:13.979664 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:13.979632 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:22:13.980123 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:13.979922 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:22:13.980800 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:13.980784 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zwn59" Jun 09 00:22:14.220649 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.220620 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.278093 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.277996 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8g5q\" (UniqueName: \"kubernetes.io/projected/6ae89e8f-6d16-4d28-a62b-3de6490d30d2-kube-api-access-b8g5q\") pod \"6ae89e8f-6d16-4d28-a62b-3de6490d30d2\" (UID: \"6ae89e8f-6d16-4d28-a62b-3de6490d30d2\") " Jun 09 00:22:14.280373 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.280340 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae89e8f-6d16-4d28-a62b-3de6490d30d2-kube-api-access-b8g5q" (OuterVolumeSpecName: "kube-api-access-b8g5q") pod "6ae89e8f-6d16-4d28-a62b-3de6490d30d2" (UID: "6ae89e8f-6d16-4d28-a62b-3de6490d30d2"). InnerVolumeSpecName "kube-api-access-b8g5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:22:14.379503 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.379458 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8g5q\" (UniqueName: \"kubernetes.io/projected/6ae89e8f-6d16-4d28-a62b-3de6490d30d2-kube-api-access-b8g5q\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:22:14.421164 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.421129 2550 generic.go:356] "Generic (PLEG): container finished" podID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerID="1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572" exitCode=143 Jun 09 00:22:14.421361 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.421179 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.421361 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.421213 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6ae89e8f-6d16-4d28-a62b-3de6490d30d2","Type":"ContainerDied","Data":"1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572"} Jun 09 00:22:14.421361 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.421272 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6ae89e8f-6d16-4d28-a62b-3de6490d30d2","Type":"ContainerDied","Data":"f542f9c8ef524224ce0485f1e36d19d8c504eacfdf35aadf6cc1b49399903fd6"} Jun 09 00:22:14.421361 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.421289 2550 scope.go:117] "RemoveContainer" containerID="1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572" Jun 09 00:22:14.431483 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.431466 2550 scope.go:117] "RemoveContainer" containerID="1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572" Jun 09 00:22:14.431774 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:22:14.431743 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572\": container with ID starting with 1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572 not found: ID does not exist" containerID="1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572" Jun 09 00:22:14.431824 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.431783 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572"} err="failed to get container status \"1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572\": rpc error: code = NotFound desc = could not find container \"1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572\": container with ID starting with 1acf308d85706e74e309d23ecf2583a68d0d787bd6d2ecfabb9e7df34a437572 not found: ID does not exist" Jun 09 00:22:14.443118 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.443090 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:22:14.445823 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.445798 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:22:14.468578 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.468543 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:22:14.469067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.469047 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" Jun 09 00:22:14.469200 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.469069 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" Jun 09 00:22:14.469259 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.469225 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" containerName="keycloak" Jun 09 00:22:14.472213 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.472192 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.474647 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.474625 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wdcp9\"" Jun 09 00:22:14.474767 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.474748 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 00:22:14.474821 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.474764 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 09 00:22:14.474821 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.474807 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 00:22:14.474919 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.474850 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 00:22:14.482259 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.482236 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:22:14.581535 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.581407 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk7kg\" (UniqueName: \"kubernetes.io/projected/76c0b48d-5bea-49dd-b37c-e568b876014a-kube-api-access-zk7kg\") pod \"maas-keycloak-0\" (UID: \"76c0b48d-5bea-49dd-b37c-e568b876014a\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.581535 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.581448 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/76c0b48d-5bea-49dd-b37c-e568b876014a-test-realms\") pod \"maas-keycloak-0\" (UID: \"76c0b48d-5bea-49dd-b37c-e568b876014a\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.682032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.681992 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk7kg\" (UniqueName: \"kubernetes.io/projected/76c0b48d-5bea-49dd-b37c-e568b876014a-kube-api-access-zk7kg\") pod \"maas-keycloak-0\" (UID: \"76c0b48d-5bea-49dd-b37c-e568b876014a\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.682032 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.682035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/76c0b48d-5bea-49dd-b37c-e568b876014a-test-realms\") pod \"maas-keycloak-0\" (UID: \"76c0b48d-5bea-49dd-b37c-e568b876014a\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.682776 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.682752 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/76c0b48d-5bea-49dd-b37c-e568b876014a-test-realms\") pod \"maas-keycloak-0\" (UID: \"76c0b48d-5bea-49dd-b37c-e568b876014a\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.690073 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.690045 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk7kg\" (UniqueName: \"kubernetes.io/projected/76c0b48d-5bea-49dd-b37c-e568b876014a-kube-api-access-zk7kg\") pod \"maas-keycloak-0\" (UID: \"76c0b48d-5bea-49dd-b37c-e568b876014a\") " pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.783146 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.783106 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:14.924096 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:14.924051 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 00:22:14.926288 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:22:14.926259 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c0b48d_5bea_49dd_b37c_e568b876014a.slice/crio-c826ffca8ac3e4ee9aca236e0f8fae54743c07d128bb57e698f9745d7fae5d2c WatchSource:0}: Error finding container c826ffca8ac3e4ee9aca236e0f8fae54743c07d128bb57e698f9745d7fae5d2c: Status 404 returned error can't find the container with id c826ffca8ac3e4ee9aca236e0f8fae54743c07d128bb57e698f9745d7fae5d2c Jun 09 00:22:15.237943 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.237912 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae89e8f-6d16-4d28-a62b-3de6490d30d2" path="/var/lib/kubelet/pods/6ae89e8f-6d16-4d28-a62b-3de6490d30d2/volumes" Jun 09 00:22:15.353170 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.353136 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:22:15.353343 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.353218 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-5brzx" Jun 09 00:22:15.431481 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.431442 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"76c0b48d-5bea-49dd-b37c-e568b876014a","Type":"ContainerStarted","Data":"51a84dd06c41541dd76064eed4b2513868cd58fcd8e025478f8b699ade83c03d"} Jun 09 00:22:15.431674 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.431508 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"76c0b48d-5bea-49dd-b37c-e568b876014a","Type":"ContainerStarted","Data":"c826ffca8ac3e4ee9aca236e0f8fae54743c07d128bb57e698f9745d7fae5d2c"} Jun 09 00:22:15.449733 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.449679 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.1051415470000001 podStartE2EDuration="1.449663378s" totalImagesPullingTime="344.521831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:22:15.447272834 +0000 UTC m=+698.671670135" watchObservedRunningTime="2026-06-09 00:22:15.449663378 +0000 UTC m=+698.674060645" Jun 09 00:22:15.783784 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.783742 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:15.785803 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:15.785768 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:16.783792 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:16.783742 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:17.784071 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:17.784020 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:18.784358 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:18.784290 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:19.784062 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:19.784013 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:20.784413 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:20.784365 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:21.783595 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:21.783544 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:22.784466 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:22.784408 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:23.783526 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:23.783473 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:24.783756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:24.783714 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:24.784230 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:24.783993 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:25.784405 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:25.784351 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:26.784130 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:26.784057 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:27.784510 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:27.784459 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:28.784158 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:28.784114 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.46:9000/health/started\": dial tcp 10.134.0.46:9000: connect: connection refused" Jun 09 00:22:29.905619 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:29.905573 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 00:22:29.920734 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:29.920685 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="76c0b48d-5bea-49dd-b37c-e568b876014a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 00:22:39.911975 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:22:39.911940 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 00:23:21.899884 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.899845 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz"] Jun 09 00:23:21.902965 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.902948 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:21.906241 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.906215 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 00:23:21.906241 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.906228 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 09 00:23:21.906438 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.906270 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fkqzn\"" Jun 09 00:23:21.906438 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.906272 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 00:23:21.914147 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.914124 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz"] Jun 09 00:23:21.986969 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.986938 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:21.987157 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.986974 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:21.987157 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.987065 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:21.987157 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.987140 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzk8n\" (UniqueName: \"kubernetes.io/projected/086070b3-daa9-428b-bee9-046e05e9bb25-kube-api-access-mzk8n\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:21.987275 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.987163 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086070b3-daa9-428b-bee9-046e05e9bb25-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:21.987275 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:21.987196 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.088697 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.088661 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.088910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.088783 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.088910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.088813 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.088910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.088884 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.089135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.088927 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzk8n\" (UniqueName: \"kubernetes.io/projected/086070b3-daa9-428b-bee9-046e05e9bb25-kube-api-access-mzk8n\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.089135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.088959 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086070b3-daa9-428b-bee9-046e05e9bb25-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.089246 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.089162 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.089309 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.089279 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.089353 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.089289 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.091560 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.091539 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086070b3-daa9-428b-bee9-046e05e9bb25-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.091672 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.091568 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086070b3-daa9-428b-bee9-046e05e9bb25-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.096454 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.096431 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzk8n\" (UniqueName: \"kubernetes.io/projected/086070b3-daa9-428b-bee9-046e05e9bb25-kube-api-access-mzk8n\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz\" (UID: \"086070b3-daa9-428b-bee9-046e05e9bb25\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.216802 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.216759 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:22.366940 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.366917 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz"] Jun 09 00:23:22.369258 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:23:22.369221 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086070b3_daa9_428b_bee9_046e05e9bb25.slice/crio-b0b8bf6a1a50222583b5d88323240b77b4bf85f233467ec206f89001dcb65590 WatchSource:0}: Error finding container b0b8bf6a1a50222583b5d88323240b77b4bf85f233467ec206f89001dcb65590: Status 404 returned error can't find the container with id b0b8bf6a1a50222583b5d88323240b77b4bf85f233467ec206f89001dcb65590 Jun 09 00:23:22.828523 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:22.828491 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" event={"ID":"086070b3-daa9-428b-bee9-046e05e9bb25","Type":"ContainerStarted","Data":"b0b8bf6a1a50222583b5d88323240b77b4bf85f233467ec206f89001dcb65590"} Jun 09 00:23:27.864159 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:27.864119 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" event={"ID":"086070b3-daa9-428b-bee9-046e05e9bb25","Type":"ContainerStarted","Data":"91ff611fa8a8e81cac57ece6e1401b21670adefcbcb4e8e56465224beb35878f"} Jun 09 00:23:28.870796 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:28.870692 2550 generic.go:356] "Generic (PLEG): container finished" podID="086070b3-daa9-428b-bee9-046e05e9bb25" containerID="91ff611fa8a8e81cac57ece6e1401b21670adefcbcb4e8e56465224beb35878f" exitCode=0 Jun 09 00:23:28.870796 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:28.870770 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" event={"ID":"086070b3-daa9-428b-bee9-046e05e9bb25","Type":"ContainerDied","Data":"91ff611fa8a8e81cac57ece6e1401b21670adefcbcb4e8e56465224beb35878f"} Jun 09 00:23:30.884406 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:30.884368 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" event={"ID":"086070b3-daa9-428b-bee9-046e05e9bb25","Type":"ContainerStarted","Data":"b24924edac3b274b3051a17539dc4ecf834259f35e6fb0319718bee8d22597e8"} Jun 09 00:23:30.903636 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:30.903576 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" podStartSLOduration=2.715720482 podStartE2EDuration="9.903559774s" totalImagesPullingTime="5.89728435s" totalInitContainerRuntime="1.290554942s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:23:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:23:30.901860715 +0000 UTC m=+774.126258003" watchObservedRunningTime="2026-06-09 00:23:30.903559774 +0000 UTC m=+774.127957041" Jun 09 00:23:32.217247 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:32.217207 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:32.217690 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:32.217271 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:32.230140 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:32.230117 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:32.907307 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:32.907273 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz" Jun 09 00:23:51.667687 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.667535 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q"] Jun 09 00:23:51.671935 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.671907 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.674368 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.674344 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 09 00:23:51.683054 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.683024 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q"] Jun 09 00:23:51.773747 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.773698 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.773936 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.773764 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.773936 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.773812 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.773936 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.773868 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f02664-a00f-4d08-b0f0-d1f61aed992e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.774107 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.773958 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlxzt\" (UniqueName: \"kubernetes.io/projected/a4f02664-a00f-4d08-b0f0-d1f61aed992e-kube-api-access-qlxzt\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.774107 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.774031 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.874568 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.874534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.874737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.874574 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.874737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.874617 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.874737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.874660 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f02664-a00f-4d08-b0f0-d1f61aed992e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.874737 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.874693 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlxzt\" (UniqueName: \"kubernetes.io/projected/a4f02664-a00f-4d08-b0f0-d1f61aed992e-kube-api-access-qlxzt\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.874949 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.874762 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.875072 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.875045 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.875243 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.875096 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.875243 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.875131 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.877565 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.877531 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f02664-a00f-4d08-b0f0-d1f61aed992e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.877676 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.877587 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f02664-a00f-4d08-b0f0-d1f61aed992e-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.882216 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.882196 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlxzt\" (UniqueName: \"kubernetes.io/projected/a4f02664-a00f-4d08-b0f0-d1f61aed992e-kube-api-access-qlxzt\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q\" (UID: \"a4f02664-a00f-4d08-b0f0-d1f61aed992e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:51.984128 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:51.984068 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:23:52.144205 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:52.144181 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q"] Jun 09 00:23:52.146250 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:23:52.146222 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4f02664_a00f_4d08_b0f0_d1f61aed992e.slice/crio-1c00eaae53a5b4058fc8149b301c5134401392001e918b0ca61b4437d8c8b8b0 WatchSource:0}: Error finding container 1c00eaae53a5b4058fc8149b301c5134401392001e918b0ca61b4437d8c8b8b0: Status 404 returned error can't find the container with id 1c00eaae53a5b4058fc8149b301c5134401392001e918b0ca61b4437d8c8b8b0 Jun 09 00:23:53.003026 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:53.002990 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" event={"ID":"a4f02664-a00f-4d08-b0f0-d1f61aed992e","Type":"ContainerStarted","Data":"10e5149cd7d138dddbc77bbbbc3cc468b51e4bddf4fbe06585a9da0a25b042a3"} Jun 09 00:23:53.003469 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:53.003048 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" event={"ID":"a4f02664-a00f-4d08-b0f0-d1f61aed992e","Type":"ContainerStarted","Data":"1c00eaae53a5b4058fc8149b301c5134401392001e918b0ca61b4437d8c8b8b0"} Jun 09 00:23:54.009688 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:54.009654 2550 generic.go:356] "Generic (PLEG): container finished" podID="a4f02664-a00f-4d08-b0f0-d1f61aed992e" containerID="10e5149cd7d138dddbc77bbbbc3cc468b51e4bddf4fbe06585a9da0a25b042a3" exitCode=0 Jun 09 00:23:54.010070 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:54.009718 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" event={"ID":"a4f02664-a00f-4d08-b0f0-d1f61aed992e","Type":"ContainerDied","Data":"10e5149cd7d138dddbc77bbbbc3cc468b51e4bddf4fbe06585a9da0a25b042a3"} Jun 09 00:23:55.017496 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:55.017452 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" event={"ID":"a4f02664-a00f-4d08-b0f0-d1f61aed992e","Type":"ContainerStarted","Data":"3532dbd968dafcf547e0acf53a8518ba3318629e9ff99322b971ef4b2fb40ca6"} Jun 09 00:23:55.035889 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:23:55.035831 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" podStartSLOduration=2.4814143 podStartE2EDuration="4.035814459s" totalImagesPullingTime="186.584838ms" totalInitContainerRuntime="1.367815321s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:23:55.034222825 +0000 UTC m=+798.258620095" watchObservedRunningTime="2026-06-09 00:23:55.035814459 +0000 UTC m=+798.260211740" Jun 09 00:24:01.984262 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:01.984221 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:24:01.984670 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:01.984299 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:24:01.997016 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:01.996986 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:24:02.065248 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:02.065220 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q" Jun 09 00:24:03.277431 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.277400 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml"] Jun 09 00:24:03.282397 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.282378 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.284768 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.284743 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 09 00:24:03.296347 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.296310 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml"] Jun 09 00:24:03.395773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.395735 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.395773 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.395777 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.395998 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.395808 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlx7b\" (UniqueName: \"kubernetes.io/projected/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-kube-api-access-tlx7b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.395998 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.395930 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.395998 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.395981 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.396122 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.396042 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.496991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497033 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497334 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497058 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497334 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497113 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlx7b\" (UniqueName: \"kubernetes.io/projected/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-kube-api-access-tlx7b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497334 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497216 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497334 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497285 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497603 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497576 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497668 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497619 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.497738 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.497717 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.499620 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.499593 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.499888 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.499869 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.504645 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.504623 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlx7b\" (UniqueName: \"kubernetes.io/projected/307cccd5-25ce-4df6-b5e5-d3523a63ceb8-kube-api-access-tlx7b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml\" (UID: \"307cccd5-25ce-4df6-b5e5-d3523a63ceb8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.601307 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.601220 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:03.747186 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:03.747158 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml"] Jun 09 00:24:03.749338 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:24:03.749306 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307cccd5_25ce_4df6_b5e5_d3523a63ceb8.slice/crio-aa07ce7b3a363efd7673cbd8c2948682fa7c155872c953dfdb7837bafcbf4cf9 WatchSource:0}: Error finding container aa07ce7b3a363efd7673cbd8c2948682fa7c155872c953dfdb7837bafcbf4cf9: Status 404 returned error can't find the container with id aa07ce7b3a363efd7673cbd8c2948682fa7c155872c953dfdb7837bafcbf4cf9 Jun 09 00:24:04.066895 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:04.066857 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="b5be139b45df3941ff0669b8c7834822063383818110718c41de073c2a01caff" exitCode=1 Jun 09 00:24:04.067126 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:04.066931 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"b5be139b45df3941ff0669b8c7834822063383818110718c41de073c2a01caff"} Jun 09 00:24:04.067306 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:04.067282 2550 scope.go:117] "RemoveContainer" containerID="b5be139b45df3941ff0669b8c7834822063383818110718c41de073c2a01caff" Jun 09 00:24:04.068803 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:04.068768 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" event={"ID":"307cccd5-25ce-4df6-b5e5-d3523a63ceb8","Type":"ContainerStarted","Data":"73fb5868b280ff5a9ee5a32b17e4229696a7cafb43e4f40237972720283d9b31"} Jun 09 00:24:04.068980 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:04.068830 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" event={"ID":"307cccd5-25ce-4df6-b5e5-d3523a63ceb8","Type":"ContainerStarted","Data":"aa07ce7b3a363efd7673cbd8c2948682fa7c155872c953dfdb7837bafcbf4cf9"} Jun 09 00:24:05.078018 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:05.077974 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57"} Jun 09 00:24:05.078458 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:05.078422 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:24:06.085475 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:06.085442 2550 generic.go:356] "Generic (PLEG): container finished" podID="307cccd5-25ce-4df6-b5e5-d3523a63ceb8" containerID="73fb5868b280ff5a9ee5a32b17e4229696a7cafb43e4f40237972720283d9b31" exitCode=0 Jun 09 00:24:06.085932 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:06.085521 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" event={"ID":"307cccd5-25ce-4df6-b5e5-d3523a63ceb8","Type":"ContainerDied","Data":"73fb5868b280ff5a9ee5a32b17e4229696a7cafb43e4f40237972720283d9b31"} Jun 09 00:24:07.092986 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:07.092951 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" event={"ID":"307cccd5-25ce-4df6-b5e5-d3523a63ceb8","Type":"ContainerStarted","Data":"66350979bd61b665cf14e718efe1194bab814e0fc446bd4fa017eaaad66d34bf"} Jun 09 00:24:07.113006 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:07.112952 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" podStartSLOduration=2.540316226 podStartE2EDuration="4.112933291s" totalImagesPullingTime="176.545347ms" totalInitContainerRuntime="1.396071718s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:24:07.11056887 +0000 UTC m=+810.334966149" watchObservedRunningTime="2026-06-09 00:24:07.112933291 +0000 UTC m=+810.337330593" Jun 09 00:24:13.601779 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:13.601717 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:13.602293 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:13.601823 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:13.615409 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:13.615382 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:14.139955 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:14.139924 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml" Jun 09 00:24:16.088264 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:16.088234 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:24:35.697879 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.697847 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh"] Jun 09 00:24:35.701792 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.701772 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.704110 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.704072 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 09 00:24:35.709806 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.709784 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh"] Jun 09 00:24:35.835304 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.835256 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.835522 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.835345 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.835522 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.835378 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.835522 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.835405 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.835522 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.835481 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc5tq\" (UniqueName: \"kubernetes.io/projected/a2b094eb-1677-42ca-aa67-a6874ca283de-kube-api-access-vc5tq\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.835693 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.835548 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b094eb-1677-42ca-aa67-a6874ca283de-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.936685 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.936640 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b094eb-1677-42ca-aa67-a6874ca283de-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.936910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.936727 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.936910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.936774 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.936910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.936792 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.936910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.936824 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.936910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.936867 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc5tq\" (UniqueName: \"kubernetes.io/projected/a2b094eb-1677-42ca-aa67-a6874ca283de-kube-api-access-vc5tq\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.937241 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.937211 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.937241 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.937224 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.937329 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.937305 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.939398 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.939370 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b094eb-1677-42ca-aa67-a6874ca283de-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.939685 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.939665 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b094eb-1677-42ca-aa67-a6874ca283de-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:35.944660 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:35.944632 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc5tq\" (UniqueName: \"kubernetes.io/projected/a2b094eb-1677-42ca-aa67-a6874ca283de-kube-api-access-vc5tq\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-727jh\" (UID: \"a2b094eb-1677-42ca-aa67-a6874ca283de\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:36.013062 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:36.013013 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:24:36.163677 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:36.163647 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh"] Jun 09 00:24:36.165798 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:24:36.165755 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b094eb_1677_42ca_aa67_a6874ca283de.slice/crio-723bcbeab22a728000cf9b783fbbc6dda09db9bc758ebbbd57a20d97cb914f19 WatchSource:0}: Error finding container 723bcbeab22a728000cf9b783fbbc6dda09db9bc758ebbbd57a20d97cb914f19: Status 404 returned error can't find the container with id 723bcbeab22a728000cf9b783fbbc6dda09db9bc758ebbbd57a20d97cb914f19 Jun 09 00:24:36.257471 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:36.257436 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" event={"ID":"a2b094eb-1677-42ca-aa67-a6874ca283de","Type":"ContainerStarted","Data":"3be05e5a4fb946811a25ed7409365dfaea4da13c887fef6738f1dc08341f9973"} Jun 09 00:24:36.257690 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:24:36.257485 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" event={"ID":"a2b094eb-1677-42ca-aa67-a6874ca283de","Type":"ContainerStarted","Data":"723bcbeab22a728000cf9b783fbbc6dda09db9bc758ebbbd57a20d97cb914f19"} Jun 09 00:25:29.562304 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:29.562260 2550 generic.go:356] "Generic (PLEG): container finished" podID="a2b094eb-1677-42ca-aa67-a6874ca283de" containerID="3be05e5a4fb946811a25ed7409365dfaea4da13c887fef6738f1dc08341f9973" exitCode=0 Jun 09 00:25:29.562712 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:29.562336 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" event={"ID":"a2b094eb-1677-42ca-aa67-a6874ca283de","Type":"ContainerDied","Data":"3be05e5a4fb946811a25ed7409365dfaea4da13c887fef6738f1dc08341f9973"} Jun 09 00:25:30.570797 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:30.570757 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" event={"ID":"a2b094eb-1677-42ca-aa67-a6874ca283de","Type":"ContainerStarted","Data":"cd02a592fea40069e9a50ad164851c25a6de3d924de2674efbe367cde9b3cf1d"} Jun 09 00:25:30.588381 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:30.588318 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" podStartSLOduration=2.237110996 podStartE2EDuration="55.588297842s" totalImagesPullingTime="197.14826ms" totalInitContainerRuntime="53.154038586s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:25:30.58750083 +0000 UTC m=+893.811898099" watchObservedRunningTime="2026-06-09 00:25:30.588297842 +0000 UTC m=+893.812695131" Jun 09 00:25:36.013606 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:36.013561 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:25:36.014019 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:36.013629 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:25:36.026067 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:36.026041 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:25:36.614867 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:25:36.614841 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-727jh" Jun 09 00:26:04.765467 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:04.765432 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57" exitCode=1 Jun 09 00:26:04.765938 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:04.765504 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57"} Jun 09 00:26:04.765938 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:04.765589 2550 scope.go:117] "RemoveContainer" containerID="b5be139b45df3941ff0669b8c7834822063383818110718c41de073c2a01caff" Jun 09 00:26:04.765938 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:04.765886 2550 scope.go:117] "RemoveContainer" containerID="2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57" Jun 09 00:26:04.766178 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:26:04.766157 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:26:06.086135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:06.086058 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:26:06.086652 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:06.086570 2550 scope.go:117] "RemoveContainer" containerID="2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57" Jun 09 00:26:06.086872 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:26:06.086846 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:26:20.230173 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:20.230133 2550 scope.go:117] "RemoveContainer" containerID="2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57" Jun 09 00:26:20.861759 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:20.861724 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795"} Jun 09 00:26:20.861966 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:20.861948 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:26:31.876889 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:26:31.876855 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:27:05.781065 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:05.781024 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-b7t25"] Jun 09 00:27:05.781608 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:05.781315 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-58c6c59df9-b7t25" podUID="cc8fdde4-0c17-445a-8cae-086377aad5d1" containerName="manager" containerID="cri-o://9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4" gracePeriod=30 Jun 09 00:27:06.044702 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.044645 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:27:06.114287 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.114257 2550 generic.go:356] "Generic (PLEG): container finished" podID="cc8fdde4-0c17-445a-8cae-086377aad5d1" containerID="9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4" exitCode=0 Jun 09 00:27:06.114487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.114299 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c6c59df9-b7t25" Jun 09 00:27:06.114487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.114339 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c6c59df9-b7t25" event={"ID":"cc8fdde4-0c17-445a-8cae-086377aad5d1","Type":"ContainerDied","Data":"9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4"} Jun 09 00:27:06.114487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.114401 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c6c59df9-b7t25" event={"ID":"cc8fdde4-0c17-445a-8cae-086377aad5d1","Type":"ContainerDied","Data":"c07e0da7dd777d6e161c208021882dd9bd8189590383b4927b0391034f673fa0"} Jun 09 00:27:06.114487 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.114431 2550 scope.go:117] "RemoveContainer" containerID="9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4" Jun 09 00:27:06.123275 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.123220 2550 scope.go:117] "RemoveContainer" containerID="9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4" Jun 09 00:27:06.123517 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:27:06.123496 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4\": container with ID starting with 9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4 not found: ID does not exist" containerID="9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4" Jun 09 00:27:06.123570 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.123527 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4"} err="failed to get container status \"9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4\": rpc error: code = NotFound desc = could not find container \"9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4\": container with ID starting with 9ee2349ef88086fb5abbf4ea74e3712475fe6e616b595c8fae554858db87f1d4 not found: ID does not exist" Jun 09 00:27:06.185699 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.185660 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8fdde4-0c17-445a-8cae-086377aad5d1-webhook-cert\") pod \"cc8fdde4-0c17-445a-8cae-086377aad5d1\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " Jun 09 00:27:06.185881 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.185845 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlsq7\" (UniqueName: \"kubernetes.io/projected/cc8fdde4-0c17-445a-8cae-086377aad5d1-kube-api-access-hlsq7\") pod \"cc8fdde4-0c17-445a-8cae-086377aad5d1\" (UID: \"cc8fdde4-0c17-445a-8cae-086377aad5d1\") " Jun 09 00:27:06.187929 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.187899 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8fdde4-0c17-445a-8cae-086377aad5d1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cc8fdde4-0c17-445a-8cae-086377aad5d1" (UID: "cc8fdde4-0c17-445a-8cae-086377aad5d1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 00:27:06.188068 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.188001 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8fdde4-0c17-445a-8cae-086377aad5d1-kube-api-access-hlsq7" (OuterVolumeSpecName: "kube-api-access-hlsq7") pod "cc8fdde4-0c17-445a-8cae-086377aad5d1" (UID: "cc8fdde4-0c17-445a-8cae-086377aad5d1"). InnerVolumeSpecName "kube-api-access-hlsq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:27:06.286966 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.286925 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlsq7\" (UniqueName: \"kubernetes.io/projected/cc8fdde4-0c17-445a-8cae-086377aad5d1-kube-api-access-hlsq7\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:27:06.286966 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.286962 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8fdde4-0c17-445a-8cae-086377aad5d1-webhook-cert\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:27:06.435159 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.435049 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-b7t25"] Jun 09 00:27:06.437719 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:06.437693 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-b7t25"] Jun 09 00:27:07.209325 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.209291 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58c6c59df9-gph9j"] Jun 09 00:27:07.209760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.209746 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8fdde4-0c17-445a-8cae-086377aad5d1" containerName="manager" Jun 09 00:27:07.209760 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.209761 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8fdde4-0c17-445a-8cae-086377aad5d1" containerName="manager" Jun 09 00:27:07.209847 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.209838 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8fdde4-0c17-445a-8cae-086377aad5d1" containerName="manager" Jun 09 00:27:07.214172 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.214152 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.217126 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.217098 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 00:27:07.217268 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.217169 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wql6k\"" Jun 09 00:27:07.217268 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.217224 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 00:27:07.219212 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.219190 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-gph9j"] Jun 09 00:27:07.239817 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.239787 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8fdde4-0c17-445a-8cae-086377aad5d1" path="/var/lib/kubelet/pods/cc8fdde4-0c17-445a-8cae-086377aad5d1/volumes" Jun 09 00:27:07.295205 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.295166 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ddc\" (UniqueName: \"kubernetes.io/projected/eb1c7418-78b2-414c-9871-68985fd71537-kube-api-access-m6ddc\") pod \"maas-controller-58c6c59df9-gph9j\" (UID: \"eb1c7418-78b2-414c-9871-68985fd71537\") " pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.295395 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.295291 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb1c7418-78b2-414c-9871-68985fd71537-webhook-cert\") pod \"maas-controller-58c6c59df9-gph9j\" (UID: \"eb1c7418-78b2-414c-9871-68985fd71537\") " pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.396636 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.396596 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb1c7418-78b2-414c-9871-68985fd71537-webhook-cert\") pod \"maas-controller-58c6c59df9-gph9j\" (UID: \"eb1c7418-78b2-414c-9871-68985fd71537\") " pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.396843 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.396665 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ddc\" (UniqueName: \"kubernetes.io/projected/eb1c7418-78b2-414c-9871-68985fd71537-kube-api-access-m6ddc\") pod \"maas-controller-58c6c59df9-gph9j\" (UID: \"eb1c7418-78b2-414c-9871-68985fd71537\") " pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.399371 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.399343 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb1c7418-78b2-414c-9871-68985fd71537-webhook-cert\") pod \"maas-controller-58c6c59df9-gph9j\" (UID: \"eb1c7418-78b2-414c-9871-68985fd71537\") " pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.404664 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.404642 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ddc\" (UniqueName: \"kubernetes.io/projected/eb1c7418-78b2-414c-9871-68985fd71537-kube-api-access-m6ddc\") pod \"maas-controller-58c6c59df9-gph9j\" (UID: \"eb1c7418-78b2-414c-9871-68985fd71537\") " pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.529122 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.529027 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:07.670899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.670873 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c6c59df9-gph9j"] Jun 09 00:27:07.672629 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:27:07.672597 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1c7418_78b2_414c_9871_68985fd71537.slice/crio-a4382d23dd643b996315b87935687a984fa91a7af4deebfe59734ff9c2b7e767 WatchSource:0}: Error finding container a4382d23dd643b996315b87935687a984fa91a7af4deebfe59734ff9c2b7e767: Status 404 returned error can't find the container with id a4382d23dd643b996315b87935687a984fa91a7af4deebfe59734ff9c2b7e767 Jun 09 00:27:07.674135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:07.674115 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 00:27:08.130131 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:08.130102 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c6c59df9-gph9j" event={"ID":"eb1c7418-78b2-414c-9871-68985fd71537","Type":"ContainerStarted","Data":"a4382d23dd643b996315b87935687a984fa91a7af4deebfe59734ff9c2b7e767"} Jun 09 00:27:09.136828 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:09.136792 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c6c59df9-gph9j" event={"ID":"eb1c7418-78b2-414c-9871-68985fd71537","Type":"ContainerStarted","Data":"b5f5c9e4f3f1a5f5a4f9e6938a36c70b91648b10fc4b2710142bab58f7639b33"} Jun 09 00:27:09.137290 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:09.136982 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:27:09.154285 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:09.154232 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58c6c59df9-gph9j" podStartSLOduration=1.760449537 podStartE2EDuration="2.154214779s" totalImagesPullingTime="393.765242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:27:09.151860414 +0000 UTC m=+992.376257693" watchObservedRunningTime="2026-06-09 00:27:09.154214779 +0000 UTC m=+992.378612047" Jun 09 00:27:20.147470 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:27:20.147438 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58c6c59df9-gph9j" Jun 09 00:28:20.528192 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:20.528158 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" exitCode=1 Jun 09 00:28:20.528713 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:20.528227 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795"} Jun 09 00:28:20.528713 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:20.528295 2550 scope.go:117] "RemoveContainer" containerID="2e11ce011483f9a90aa5ffd83f1464d6f7d9e2ee5872180da1ef65fb6dc61a57" Jun 09 00:28:20.528713 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:20.528618 2550 scope.go:117] "RemoveContainer" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" Jun 09 00:28:20.528941 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:28:20.528919 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:28:20.941162 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:20.941133 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:28:21.535764 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:21.535740 2550 scope.go:117] "RemoveContainer" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" Jun 09 00:28:21.536150 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:28:21.535946 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:28:21.874656 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:21.874572 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:28:22.540492 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:22.540464 2550 scope.go:117] "RemoveContainer" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" Jun 09 00:28:22.540889 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:28:22.540682 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:28:36.229933 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:36.229900 2550 scope.go:117] "RemoveContainer" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" Jun 09 00:28:36.230430 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:28:36.230180 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:28:47.232326 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:47.232301 2550 scope.go:117] "RemoveContainer" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" Jun 09 00:28:47.675496 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:47.675414 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4"} Jun 09 00:28:47.675656 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:47.675636 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:28:58.683284 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:28:58.683250 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:30:48.337174 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:30:48.337133 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" exitCode=1 Jun 09 00:30:48.337696 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:30:48.337210 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4"} Jun 09 00:30:48.337696 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:30:48.337278 2550 scope.go:117] "RemoveContainer" containerID="e0984eaeac99374bd13f81dd0a7ae8453afacdf5a26679f017f7d355a63b2795" Jun 09 00:30:48.337696 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:30:48.337627 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:30:48.337910 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:30:48.337887 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:30:48.682382 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:30:48.682290 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:30:49.343987 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:30:49.343958 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:30:49.344502 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:30:49.344319 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:31:00.941175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:00.941139 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:31:00.941595 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:00.941531 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:31:00.941727 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:31:00.941706 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:31:11.236416 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:11.236386 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:31:11.236802 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:31:11.236600 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:31:22.229725 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:22.229642 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:31:22.230119 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:31:22.229853 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:31:33.230561 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:33.230527 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:31:33.598282 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:33.598186 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf"} Jun 09 00:31:33.598453 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:33.598416 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:31:44.605227 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:31:44.605191 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:33:34.266359 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:34.266325 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" exitCode=1 Jun 09 00:33:34.266871 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:34.266403 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf"} Jun 09 00:33:34.266871 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:34.266479 2550 scope.go:117] "RemoveContainer" containerID="9cceada26d8105a97c1e2fe65b1ac49b3bc086fa627a7bc8f6ee9dbea440a7a4" Jun 09 00:33:34.266871 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:34.266732 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:33:34.267008 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:33:34.266986 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:33:34.603502 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:34.603420 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:33:35.273899 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:35.273869 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:33:35.274345 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:33:35.274045 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:33:40.941153 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:40.941107 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:33:40.941756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:40.941662 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:33:40.941951 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:33:40.941926 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:33:55.230463 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:33:55.230427 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:33:55.230863 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:33:55.230641 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:34:09.230538 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:34:09.230505 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:34:09.230944 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:34:09.230707 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:34:22.229865 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:34:22.229792 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:34:22.230345 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:34:22.230021 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:34:37.233261 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:34:37.233231 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:34:37.233656 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:34:37.233460 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:34:50.230269 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:34:50.230233 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:34:50.230663 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:34:50.230473 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:35:01.230515 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:35:01.230477 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:35:01.764037 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:35:01.763998 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d"} Jun 09 00:35:01.764250 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:35:01.764228 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:35:12.772313 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:35:12.772268 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:37:01.437574 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:01.437540 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" exitCode=1 Jun 09 00:37:01.438059 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:01.437617 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d"} Jun 09 00:37:01.438059 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:01.437695 2550 scope.go:117] "RemoveContainer" containerID="a7e76881cacede0151e406eef668ff842607404092b4e86e5272af88f61b0ccf" Jun 09 00:37:01.438192 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:01.438061 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:01.438417 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:01.438395 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:37:02.769826 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:02.769786 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:37:02.770248 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:02.770120 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:02.770362 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:02.770342 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:37:17.236572 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:17.236498 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:17.237005 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:17.236714 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:37:20.940810 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:20.940765 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:37:20.941264 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:20.941194 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:20.941459 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:20.941438 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:37:32.229700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:32.229668 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:32.230204 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:32.229878 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:37:34.104732 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.104700 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2"] Jun 09 00:37:34.105246 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.104951 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" podUID="a57b8430-8248-42fe-a454-3dcf41a832e3" containerName="manager" containerID="cri-o://dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5" gracePeriod=10 Jun 09 00:37:34.368166 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.368104 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:37:34.478720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.478678 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfll2\" (UniqueName: \"kubernetes.io/projected/a57b8430-8248-42fe-a454-3dcf41a832e3-kube-api-access-hfll2\") pod \"a57b8430-8248-42fe-a454-3dcf41a832e3\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " Jun 09 00:37:34.478902 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.478747 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a57b8430-8248-42fe-a454-3dcf41a832e3-extensions-socket-volume\") pod \"a57b8430-8248-42fe-a454-3dcf41a832e3\" (UID: \"a57b8430-8248-42fe-a454-3dcf41a832e3\") " Jun 09 00:37:34.479160 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.479137 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a57b8430-8248-42fe-a454-3dcf41a832e3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a57b8430-8248-42fe-a454-3dcf41a832e3" (UID: "a57b8430-8248-42fe-a454-3dcf41a832e3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 00:37:34.481043 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.481008 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a57b8430-8248-42fe-a454-3dcf41a832e3-kube-api-access-hfll2" (OuterVolumeSpecName: "kube-api-access-hfll2") pod "a57b8430-8248-42fe-a454-3dcf41a832e3" (UID: "a57b8430-8248-42fe-a454-3dcf41a832e3"). InnerVolumeSpecName "kube-api-access-hfll2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 00:37:34.580240 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.580210 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfll2\" (UniqueName: \"kubernetes.io/projected/a57b8430-8248-42fe-a454-3dcf41a832e3-kube-api-access-hfll2\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:37:34.580240 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.580235 2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a57b8430-8248-42fe-a454-3dcf41a832e3-extensions-socket-volume\") on node \"ip-10-0-143-86.ec2.internal\" DevicePath \"\"" Jun 09 00:37:34.627175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.627059 2550 generic.go:356] "Generic (PLEG): container finished" podID="a57b8430-8248-42fe-a454-3dcf41a832e3" containerID="dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5" exitCode=0 Jun 09 00:37:34.627175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.627123 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" Jun 09 00:37:34.627175 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.627141 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" event={"ID":"a57b8430-8248-42fe-a454-3dcf41a832e3","Type":"ContainerDied","Data":"dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5"} Jun 09 00:37:34.627411 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.627194 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2" event={"ID":"a57b8430-8248-42fe-a454-3dcf41a832e3","Type":"ContainerDied","Data":"9837c42512bf305d73c6d6eeb4589a949989a45f69392fb94b7a1274e056d5b1"} Jun 09 00:37:34.627411 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.627216 2550 scope.go:117] "RemoveContainer" containerID="dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5" Jun 09 00:37:34.638037 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.638019 2550 scope.go:117] "RemoveContainer" containerID="dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5" Jun 09 00:37:34.638334 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:34.638315 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5\": container with ID starting with dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5 not found: ID does not exist" containerID="dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5" Jun 09 00:37:34.638386 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.638345 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5"} err="failed to get container status \"dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5\": rpc error: code = NotFound desc = could not find container \"dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5\": container with ID starting with dee627bf87fabdf252aec3e6929f868c948947924ceb26b47bd287fb639dd6c5 not found: ID does not exist" Jun 09 00:37:34.650557 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.650532 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2"] Jun 09 00:37:34.652555 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:34.652536 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jfpm2"] Jun 09 00:37:35.236699 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:35.236667 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a57b8430-8248-42fe-a454-3dcf41a832e3" path="/var/lib/kubelet/pods/a57b8430-8248-42fe-a454-3dcf41a832e3/volumes" Jun 09 00:37:45.230556 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:45.230516 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:45.231017 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:45.230773 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:37:59.230028 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:37:59.229990 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:37:59.230616 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:37:59.230313 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:38:12.229697 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:12.229669 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:38:12.230124 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:38:12.229888 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:38:26.230612 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:26.230575 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:38:26.231105 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:38:26.230825 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:38:40.293345 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.293303 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872"] Jun 09 00:38:40.293837 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.293766 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a57b8430-8248-42fe-a454-3dcf41a832e3" containerName="manager" Jun 09 00:38:40.293837 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.293779 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a57b8430-8248-42fe-a454-3dcf41a832e3" containerName="manager" Jun 09 00:38:40.293910 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.293870 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a57b8430-8248-42fe-a454-3dcf41a832e3" containerName="manager" Jun 09 00:38:40.297240 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.297216 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.299914 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.299885 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rvk2b\"" Jun 09 00:38:40.311247 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.311217 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872"] Jun 09 00:38:40.356153 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.356117 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc8hg\" (UniqueName: \"kubernetes.io/projected/563e62cc-1fcb-4c55-8a36-01f1edf07ddb-kube-api-access-qc8hg\") pod \"kuadrant-operator-controller-manager-86878c769f-kr872\" (UID: \"563e62cc-1fcb-4c55-8a36-01f1edf07ddb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.356346 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.356161 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/563e62cc-1fcb-4c55-8a36-01f1edf07ddb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kr872\" (UID: \"563e62cc-1fcb-4c55-8a36-01f1edf07ddb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.457421 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.457376 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc8hg\" (UniqueName: \"kubernetes.io/projected/563e62cc-1fcb-4c55-8a36-01f1edf07ddb-kube-api-access-qc8hg\") pod \"kuadrant-operator-controller-manager-86878c769f-kr872\" (UID: \"563e62cc-1fcb-4c55-8a36-01f1edf07ddb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.457421 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.457424 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/563e62cc-1fcb-4c55-8a36-01f1edf07ddb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kr872\" (UID: \"563e62cc-1fcb-4c55-8a36-01f1edf07ddb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.457830 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.457814 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/563e62cc-1fcb-4c55-8a36-01f1edf07ddb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kr872\" (UID: \"563e62cc-1fcb-4c55-8a36-01f1edf07ddb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.465469 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.465441 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc8hg\" (UniqueName: \"kubernetes.io/projected/563e62cc-1fcb-4c55-8a36-01f1edf07ddb-kube-api-access-qc8hg\") pod \"kuadrant-operator-controller-manager-86878c769f-kr872\" (UID: \"563e62cc-1fcb-4c55-8a36-01f1edf07ddb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.609511 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.609413 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:40.762427 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.762398 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872"] Jun 09 00:38:40.764865 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:38:40.764831 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563e62cc_1fcb_4c55_8a36_01f1edf07ddb.slice/crio-54b43b715b543856aa5d46d8c9a8b04b3b41abe6b54d27f249e84c1ae1dfa91b WatchSource:0}: Error finding container 54b43b715b543856aa5d46d8c9a8b04b3b41abe6b54d27f249e84c1ae1dfa91b: Status 404 returned error can't find the container with id 54b43b715b543856aa5d46d8c9a8b04b3b41abe6b54d27f249e84c1ae1dfa91b Jun 09 00:38:40.988212 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.988172 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" event={"ID":"563e62cc-1fcb-4c55-8a36-01f1edf07ddb","Type":"ContainerStarted","Data":"68ed66a147313366652eab30fbe501a30b4064c36033b2367b7d08b00962b8f9"} Jun 09 00:38:40.988443 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.988235 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" event={"ID":"563e62cc-1fcb-4c55-8a36-01f1edf07ddb","Type":"ContainerStarted","Data":"54b43b715b543856aa5d46d8c9a8b04b3b41abe6b54d27f249e84c1ae1dfa91b"} Jun 09 00:38:40.988443 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:40.988281 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:41.008345 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:41.008279 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" podStartSLOduration=1.008263384 podStartE2EDuration="1.008263384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 00:38:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:38:41.005538142 +0000 UTC m=+1684.229935436" watchObservedRunningTime="2026-06-09 00:38:41.008263384 +0000 UTC m=+1684.232660651" Jun 09 00:38:41.230787 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:41.230749 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:38:41.231020 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:38:41.230996 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:38:51.995752 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:51.995660 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kr872" Jun 09 00:38:55.230238 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:38:55.230201 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:38:55.230706 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:38:55.230446 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:39:09.230715 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:39:09.230681 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:39:09.231164 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:39:09.230906 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:39:22.230111 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:39:22.230051 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:39:22.230586 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:39:22.230333 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:39:36.229900 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:39:36.229865 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:39:36.230407 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:39:36.230137 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:39:51.229754 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:39:51.229719 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:39:51.405295 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:39:51.405231 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9"} Jun 09 00:39:51.405777 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:39:51.405746 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:40:02.413748 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:40:02.413712 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:41:52.078460 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:41:52.078424 2550 generic.go:356] "Generic (PLEG): container finished" podID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" exitCode=1 Jun 09 00:41:52.078942 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:41:52.078494 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerDied","Data":"34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9"} Jun 09 00:41:52.078942 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:41:52.078583 2550 scope.go:117] "RemoveContainer" containerID="0dd40adf349149b8f58799dc7db5f5d4b508bfdda68a09eb506d1cffd2618e8d" Jun 09 00:41:52.079106 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:41:52.079020 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:41:52.079306 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:41:52.079285 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:41:52.412403 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:41:52.412315 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:41:53.085883 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:41:53.085858 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:41:53.086406 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:41:53.086153 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:42:00.941191 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:42:00.941138 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:42:00.941672 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:42:00.941489 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:42:00.941759 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:42:00.941738 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:42:15.230404 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:42:15.230366 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:42:15.230977 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:42:15.230619 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:42:27.234409 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:42:27.234375 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:42:27.234929 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:42:27.234609 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:42:41.230264 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:42:41.230228 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:42:41.230745 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:42:41.230483 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:42:52.229672 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:42:52.229646 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:42:52.230138 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:42:52.229845 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:43:07.234137 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:43:07.234104 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:43:07.236367 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:43:07.234429 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:43:20.230923 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:43:20.230839 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:43:20.231347 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:43:20.231038 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:43:32.229557 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:43:32.229528 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:43:32.230013 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:43:32.229749 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:43:43.230183 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:43:43.230151 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:43:43.230657 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:43:43.230323 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:43:56.230501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:43:56.230465 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:43:56.230956 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:43:56.230719 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:44:09.237575 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:44:09.237539 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:44:09.238004 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:44:09.237724 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:44:23.230720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:44:23.230684 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:44:23.231156 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:44:23.231000 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:44:34.230535 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:44:34.230501 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:44:34.231147 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:44:34.230786 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:44:48.230776 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:44:48.230690 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:44:48.231258 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:44:48.230878 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:45:01.230541 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:45:01.230502 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:45:01.231035 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:45:01.230769 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:45:15.230364 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:45:15.230332 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:45:15.230769 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:45:15.230580 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:45:26.230602 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:45:26.230558 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:45:26.231002 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:45:26.230805 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:45:39.230758 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:45:39.230718 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:45:39.231377 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:45:39.230954 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:45:54.230561 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:45:54.230527 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:45:54.231165 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:45:54.230763 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:46:06.230327 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:06.230290 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:46:06.230788 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:46:06.230471 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:46:20.230721 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:20.230628 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:46:20.231230 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:46:20.230897 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:46:31.230205 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:31.230163 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:46:31.230751 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:46:31.230399 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:46:45.230013 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:45.229974 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:46:45.230568 ip-10-0-143-86 kubenswrapper[2550]: E0609 00:46:45.230254 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=payload-processing pod=payload-processing-7c6c859784-q6lhg_openshift-ingress(592a56c4-98b0-40cc-9ad7-d47627b8e6df)\"" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" podUID="592a56c4-98b0-40cc-9ad7-d47627b8e6df" Jun 09 00:46:57.232482 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:57.232451 2550 scope.go:117] "RemoveContainer" containerID="34584bd4102880c3bc9d7b56ce9114ac646dbc3d74edd9a7971549b37d373da9" Jun 09 00:46:57.236070 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:57.236048 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 09 00:46:57.783213 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:57.783176 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" event={"ID":"592a56c4-98b0-40cc-9ad7-d47627b8e6df","Type":"ContainerStarted","Data":"bf7ade44c052bc95b0795469e9a2e97a6ae35def54a4860d2480aad19ed252cd"} Jun 09 00:46:57.783428 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:46:57.783389 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:47:08.791250 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:08.791218 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-q6lhg" Jun 09 00:47:56.056870 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:56.056790 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-77774b9544-tlt67_c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706/nginx/0.log" Jun 09 00:47:56.407047 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:56.406959 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58c6c59df9-gph9j_eb1c7418-78b2-414c-9871-68985fd71537/manager/0.log" Jun 09 00:47:56.998238 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:56.998208 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8d497bc9-nzjv5_9d915f03-acb8-496b-ae92-506da5515857/copy-manifests/0.log" Jun 09 00:47:57.011395 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:57.011363 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8d497bc9-nzjv5_9d915f03-acb8-496b-ae92-506da5515857/manager/0.log" Jun 09 00:47:58.405029 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:58.404988 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-947cs_47f50faf-df9b-48c6-a748-23daa9f15e76/manager/0.log" Jun 09 00:47:58.517277 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:58.517244 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jgwmb_26293972-949d-4dd0-b511-8da4ab08d7b1/manager/0.log" Jun 09 00:47:58.631696 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:58.631661 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-h9z5c_6cd5181a-d2e2-4f9e-885b-e3df8df51424/kuadrant-console-plugin/0.log" Jun 09 00:47:58.750365 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:58.750331 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wj4g8_a57d7aca-4044-4927-a7ca-177025e9f57f/registry-server/0.log" Jun 09 00:47:58.874605 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:58.874571 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kr872_563e62cc-1fcb-4c55-8a36-01f1edf07ddb/manager/0.log" Jun 09 00:47:59.469573 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:59.469543 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc5pcjh_e467d419-13df-4ca3-a5c0-2c520ed5ac73/istio-proxy/0.log" Jun 09 00:47:59.937405 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:47:59.937319 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/1.log" Jun 09 00:48:00.049613 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.049576 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-5brzx_b94d3fe5-f1d0-4ff4-9f5d-a973028d8303/payload-pre-processing/0.log" Jun 09 00:48:00.163059 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.163023 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/7.log" Jun 09 00:48:00.614700 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.614669 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz_086070b3-daa9-428b-bee9-046e05e9bb25/storage-initializer/0.log" Jun 09 00:48:00.622312 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.622282 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-f8prz_086070b3-daa9-428b-bee9-046e05e9bb25/main/0.log" Jun 09 00:48:00.853834 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.853803 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q_a4f02664-a00f-4d08-b0f0-d1f61aed992e/storage-initializer/0.log" Jun 09 00:48:00.861756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.861735 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-wth2q_a4f02664-a00f-4d08-b0f0-d1f61aed992e/main/0.log" Jun 09 00:48:00.981495 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.981468 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml_307cccd5-25ce-4df6-b5e5-d3523a63ceb8/storage-initializer/0.log" Jun 09 00:48:00.989190 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:00.989163 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6r8ml_307cccd5-25ce-4df6-b5e5-d3523a63ceb8/main/0.log" Jun 09 00:48:01.221694 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:01.221663 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-727jh_a2b094eb-1677-42ca-aa67-a6874ca283de/storage-initializer/0.log" Jun 09 00:48:01.229717 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:01.229692 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-727jh_a2b094eb-1677-42ca-aa67-a6874ca283de/main/0.log" Jun 09 00:48:08.037978 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:08.037941 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x8h82_f62773d3-957a-4951-8974-a26ee5ebd3cb/global-pull-secret-syncer/0.log" Jun 09 00:48:08.109501 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:08.109468 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jzrmb_4b633971-bf33-4527-b021-b8f97a6b1f69/konnectivity-agent/0.log" Jun 09 00:48:08.204374 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:08.204343 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-86.ec2.internal_b2ca2fd9e0ea5addc9b16951a720ff81/haproxy/0.log" Jun 09 00:48:13.258973 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:13.258938 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-947cs_47f50faf-df9b-48c6-a748-23daa9f15e76/manager/0.log" Jun 09 00:48:13.291894 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:13.291853 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jgwmb_26293972-949d-4dd0-b511-8da4ab08d7b1/manager/0.log" Jun 09 00:48:13.322650 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:13.322624 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-h9z5c_6cd5181a-d2e2-4f9e-885b-e3df8df51424/kuadrant-console-plugin/0.log" Jun 09 00:48:13.369746 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:13.369708 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wj4g8_a57d7aca-4044-4927-a7ca-177025e9f57f/registry-server/0.log" Jun 09 00:48:13.441366 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:13.441336 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kr872_563e62cc-1fcb-4c55-8a36-01f1edf07ddb/manager/0.log" Jun 09 00:48:15.403244 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.403196 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-txxtz_b5575c1d-f464-4425-aced-8212ec8efe84/kube-state-metrics/0.log" Jun 09 00:48:15.419580 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.419550 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-txxtz_b5575c1d-f464-4425-aced-8212ec8efe84/kube-rbac-proxy-main/0.log" Jun 09 00:48:15.438855 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.438826 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-txxtz_b5575c1d-f464-4425-aced-8212ec8efe84/kube-rbac-proxy-self/0.log" Jun 09 00:48:15.467824 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.467796 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c58b74978-p5dlr_10230a7c-9256-4478-8734-3aa4b1f7b4ef/metrics-server/0.log" Jun 09 00:48:15.492254 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.492220 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-8xwrb_7e4047ca-3d38-4a2a-93e8-038c442c2900/monitoring-plugin/0.log" Jun 09 00:48:15.522481 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.522451 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4xx8m_e89141d3-30f6-4aa3-8c9e-c2fa46c8c920/node-exporter/0.log" Jun 09 00:48:15.547306 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.547277 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4xx8m_e89141d3-30f6-4aa3-8c9e-c2fa46c8c920/kube-rbac-proxy/0.log" Jun 09 00:48:15.569803 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.569725 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4xx8m_e89141d3-30f6-4aa3-8c9e-c2fa46c8c920/init-textfile/0.log" Jun 09 00:48:15.818597 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.818565 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/prometheus/0.log" Jun 09 00:48:15.837135 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.837054 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/config-reloader/0.log" Jun 09 00:48:15.856148 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.856119 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/thanos-sidecar/0.log" Jun 09 00:48:15.877423 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.877390 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/kube-rbac-proxy-web/0.log" Jun 09 00:48:15.925977 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.925949 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/kube-rbac-proxy/0.log" Jun 09 00:48:15.946194 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.946159 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/kube-rbac-proxy-thanos/0.log" Jun 09 00:48:15.965444 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.965393 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e69f0b3-15c3-4d0f-bc64-7c03006a275c/init-config-reloader/0.log" Jun 09 00:48:15.993898 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:15.993864 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-rc55j_88d98b0f-1cf2-4794-8301-e3661a6048be/prometheus-operator/0.log" Jun 09 00:48:16.010733 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:16.010706 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-rc55j_88d98b0f-1cf2-4794-8301-e3661a6048be/kube-rbac-proxy/0.log" Jun 09 00:48:16.070580 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:16.070548 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-88f54cdc5-fhw7p_09e4639f-ab50-4369-98c3-3165b18c3cc7/telemeter-client/0.log" Jun 09 00:48:16.090336 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:16.090260 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-88f54cdc5-fhw7p_09e4639f-ab50-4369-98c3-3165b18c3cc7/reload/0.log" Jun 09 00:48:16.113669 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:16.113638 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-88f54cdc5-fhw7p_09e4639f-ab50-4369-98c3-3165b18c3cc7/kube-rbac-proxy/0.log" Jun 09 00:48:17.632794 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:17.632766 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-bfx2d_10b97160-eef2-4d08-8c87-8414b5c990a5/networking-console-plugin/0.log" Jun 09 00:48:18.793251 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:18.793223 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8694df9b48-4pvx8_0d8e401f-240a-49c2-b3ef-3b96b838071f/console/0.log" Jun 09 00:48:19.718628 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.718587 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx"] Jun 09 00:48:19.722709 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.722679 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.725177 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.725147 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2rrv6\"/\"openshift-service-ca.crt\"" Jun 09 00:48:19.725993 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.725965 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2rrv6\"/\"default-dockercfg-86trh\"" Jun 09 00:48:19.726136 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.726021 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2rrv6\"/\"kube-root-ca.crt\"" Jun 09 00:48:19.732723 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.732689 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx"] Jun 09 00:48:19.787459 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.787418 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-sys\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.787459 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.787461 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-lib-modules\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.787701 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.787484 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-podres\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.787701 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.787569 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg8kx\" (UniqueName: \"kubernetes.io/projected/ec58befb-0306-4040-8f71-abe673ae094a-kube-api-access-rg8kx\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.787701 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.787626 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-proc\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889055 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889018 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg8kx\" (UniqueName: \"kubernetes.io/projected/ec58befb-0306-4040-8f71-abe673ae094a-kube-api-access-rg8kx\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889071 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-proc\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889232 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-sys\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889263 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-lib-modules\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889277 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-sys\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889232 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-proc\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889284 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-podres\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889401 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-podres\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.889500 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.889418 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec58befb-0306-4040-8f71-abe673ae094a-lib-modules\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:19.896800 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:19.896764 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg8kx\" (UniqueName: \"kubernetes.io/projected/ec58befb-0306-4040-8f71-abe673ae094a-kube-api-access-rg8kx\") pod \"perf-node-gather-daemonset-7ssqx\" (UID: \"ec58befb-0306-4040-8f71-abe673ae094a\") " pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:20.035133 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:20.034998 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:20.187604 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:20.187573 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx"] Jun 09 00:48:20.189815 ip-10-0-143-86 kubenswrapper[2550]: W0609 00:48:20.189782 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podec58befb_0306_4040_8f71_abe673ae094a.slice/crio-555fefad6afb7422507d96d35d2b156f63eb42bc6c6d3c775a7b661baf16cd0d WatchSource:0}: Error finding container 555fefad6afb7422507d96d35d2b156f63eb42bc6c6d3c775a7b661baf16cd0d: Status 404 returned error can't find the container with id 555fefad6afb7422507d96d35d2b156f63eb42bc6c6d3c775a7b661baf16cd0d Jun 09 00:48:20.246099 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:20.246054 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" event={"ID":"ec58befb-0306-4040-8f71-abe673ae094a","Type":"ContainerStarted","Data":"555fefad6afb7422507d96d35d2b156f63eb42bc6c6d3c775a7b661baf16cd0d"} Jun 09 00:48:20.693062 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:20.692982 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pd9ph_541a23f9-36b8-4b1f-a7a3-254f348ec588/dns/0.log" Jun 09 00:48:20.713912 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:20.713874 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pd9ph_541a23f9-36b8-4b1f-a7a3-254f348ec588/kube-rbac-proxy/0.log" Jun 09 00:48:20.804579 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:20.804547 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g97wc_dc81db3e-2bb4-42bc-9b8d-c07d09d3226e/dns-node-resolver/0.log" Jun 09 00:48:21.254056 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:21.254013 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" event={"ID":"ec58befb-0306-4040-8f71-abe673ae094a","Type":"ContainerStarted","Data":"5c67e3ff60fd3cd91a6addba779b641416d71c3d2424b3081cfa3efcdad9e3eb"} Jun 09 00:48:21.254513 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:21.254246 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:21.273527 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:21.273466 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" podStartSLOduration=2.273446438 podStartE2EDuration="2.273446438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 00:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 00:48:21.270729939 +0000 UTC m=+2264.495127207" watchObservedRunningTime="2026-06-09 00:48:21.273446438 +0000 UTC m=+2264.497843706" Jun 09 00:48:21.487203 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:21.487172 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nj5z7_3e0a6921-f28b-42f1-a2c9-05ed1e65c4de/node-ca/0.log" Jun 09 00:48:22.405327 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:22.405293 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc5pcjh_e467d419-13df-4ca3-a5c0-2c520ed5ac73/istio-proxy/0.log" Jun 09 00:48:22.533707 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:22.533667 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/0.log" Jun 09 00:48:22.563725 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:22.563688 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/1.log" Jun 09 00:48:22.585473 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:22.585437 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-5brzx_b94d3fe5-f1d0-4ff4-9f5d-a973028d8303/payload-pre-processing/0.log" Jun 09 00:48:22.608353 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:22.608327 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/7.log" Jun 09 00:48:22.608720 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:22.608702 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/6.log" Jun 09 00:48:23.225139 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:23.225107 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gtn7g_9bf29450-7699-4d9e-8505-87f6b8688f9c/serve-healthcheck-canary/0.log" Jun 09 00:48:23.751637 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:23.751592 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-jk8dr_2556458f-8fa9-489e-86f4-b52a169ef0dc/insights-operator/0.log" Jun 09 00:48:23.752213 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:23.752063 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-jk8dr_2556458f-8fa9-489e-86f4-b52a169ef0dc/insights-operator/1.log" Jun 09 00:48:23.840592 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:23.840560 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5ktt_e767f1c2-5f25-4bd2-bcf4-571000193825/kube-rbac-proxy/0.log" Jun 09 00:48:23.861796 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:23.861768 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5ktt_e767f1c2-5f25-4bd2-bcf4-571000193825/exporter/0.log" Jun 09 00:48:23.882965 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:23.882914 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5ktt_e767f1c2-5f25-4bd2-bcf4-571000193825/extractor/0.log" Jun 09 00:48:26.107445 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:26.107417 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-77774b9544-tlt67_c0f08b8f-c24e-47e6-9ee1-dd8cfd4e9706/nginx/0.log" Jun 09 00:48:26.267756 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:26.267720 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58c6c59df9-gph9j_eb1c7418-78b2-414c-9871-68985fd71537/manager/0.log" Jun 09 00:48:26.524432 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:26.524390 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8d497bc9-nzjv5_9d915f03-acb8-496b-ae92-506da5515857/manager/0.log" Jun 09 00:48:26.540919 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:26.540893 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8d497bc9-nzjv5_9d915f03-acb8-496b-ae92-506da5515857/copy-manifests/0.log" Jun 09 00:48:27.271207 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:27.271179 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2rrv6/perf-node-gather-daemonset-7ssqx" Jun 09 00:48:27.946973 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:27.946935 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-844bd95cf6-fpmxz_5fc22ee6-96ae-419c-9472-cd8c170e0bfd/manager/0.log" Jun 09 00:48:32.945838 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:32.945798 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc5pcjh_e467d419-13df-4ca3-a5c0-2c520ed5ac73/istio-proxy/0.log" Jun 09 00:48:33.056577 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:33.056548 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/0.log" Jun 09 00:48:33.092961 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:33.092926 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/1.log" Jun 09 00:48:33.102747 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:33.102723 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-5brzx_b94d3fe5-f1d0-4ff4-9f5d-a973028d8303/payload-pre-processing/0.log" Jun 09 00:48:33.119621 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:33.119593 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/7.log" Jun 09 00:48:33.119926 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:33.119911 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/6.log" Jun 09 00:48:34.324300 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.324271 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc5pcjh_e467d419-13df-4ca3-a5c0-2c520ed5ac73/istio-proxy/0.log" Jun 09 00:48:34.437526 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.437482 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/0.log" Jun 09 00:48:34.473375 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.473334 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/1.log" Jun 09 00:48:34.485726 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.485699 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-5brzx_b94d3fe5-f1d0-4ff4-9f5d-a973028d8303/payload-pre-processing/0.log" Jun 09 00:48:34.503486 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.503458 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/7.log" Jun 09 00:48:34.503814 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.503793 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/6.log" Jun 09 00:48:34.912230 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.912203 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/kube-multus-additional-cni-plugins/0.log" Jun 09 00:48:34.933832 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.933806 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/egress-router-binary-copy/0.log" Jun 09 00:48:34.954934 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.954907 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/cni-plugins/0.log" Jun 09 00:48:34.974223 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.974199 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/bond-cni-plugin/0.log" Jun 09 00:48:34.993945 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:34.993916 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/routeoverride-cni/0.log" Jun 09 00:48:35.013844 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:35.013789 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/whereabouts-cni-bincopy/0.log" Jun 09 00:48:35.037877 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:35.037844 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rt7rv_ccd3a06c-34ce-446e-8593-860e6a8eb7c6/whereabouts-cni/0.log" Jun 09 00:48:35.344417 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:35.344377 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g9lwf_b286762b-0f99-494b-9e7c-d89fbd551c00/kube-multus/0.log" Jun 09 00:48:35.445491 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:35.445458 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vx6qc_c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0/network-metrics-daemon/0.log" Jun 09 00:48:35.465268 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:35.465230 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vx6qc_c9a53b44-c4c8-4e51-ae49-ee0ddda0f5f0/kube-rbac-proxy/0.log" Jun 09 00:48:35.988783 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:35.988751 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc5pcjh_e467d419-13df-4ca3-a5c0-2c520ed5ac73/istio-proxy/0.log" Jun 09 00:48:36.092972 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.092850 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/0.log" Jun 09 00:48:36.126051 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.126017 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zwn59_739ee8d3-2160-452a-be22-14f55d145173/istio-proxy/1.log" Jun 09 00:48:36.136584 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.136550 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-5brzx_b94d3fe5-f1d0-4ff4-9f5d-a973028d8303/payload-pre-processing/0.log" Jun 09 00:48:36.151990 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.151941 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/7.log" Jun 09 00:48:36.152269 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.152246 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-q6lhg_592a56c4-98b0-40cc-9ad7-d47627b8e6df/payload-processing/6.log" Jun 09 00:48:36.837826 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.837789 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/ovn-controller/0.log" Jun 09 00:48:36.857430 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.857403 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/ovn-acl-logging/0.log" Jun 09 00:48:36.879768 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.879735 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/kube-rbac-proxy-node/0.log" Jun 09 00:48:36.902631 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.902605 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/kube-rbac-proxy-ovn-metrics/0.log" Jun 09 00:48:36.918647 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.918619 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/northd/0.log" Jun 09 00:48:36.937740 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.937697 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/nbdb/0.log" Jun 09 00:48:36.956185 ip-10-0-143-86 kubenswrapper[2550]: I0609 00:48:36.956143 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wks97_dd74db37-2abc-4b24-8362-f410ec8c144a/sbdb/0.log"