Jun 03 16:41:19.177256 ip-10-0-136-251 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 03 16:41:19.177268 ip-10-0-136-251 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 03 16:41:19.177277 ip-10-0-136-251 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 03 16:41:19.177572 ip-10-0-136-251 systemd[1]: Failed to start Kubernetes Kubelet. Jun 03 16:41:29.177897 ip-10-0-136-251 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 03 16:41:29.177913 ip-10-0-136-251 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 922e4e04a8a7483f973bbf685a10e22c -- Jun 03 16:43:47.520894 ip-10-0-136-251 systemd[1]: Starting Kubernetes Kubelet... Jun 03 16:43:47.873020 ip-10-0-136-251 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:43:47.873020 ip-10-0-136-251 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 16:43:47.873020 ip-10-0-136-251 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:43:47.873020 ip-10-0-136-251 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 16:43:47.873020 ip-10-0-136-251 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:43:47.875495 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.875409 2578 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 16:43:47.882639 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882616 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:43:47.882639 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882636 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:43:47.882639 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882642 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882652 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882656 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882660 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882664 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882668 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882673 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882677 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882681 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882685 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882689 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882693 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882697 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882705 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882710 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882715 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882721 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882725 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882730 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882734 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:43:47.882841 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882738 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882742 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882746 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882750 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882754 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882762 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882766 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882771 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882775 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882780 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882784 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882789 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882793 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882798 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882803 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882808 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882815 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882822 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882830 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882835 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:43:47.883667 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882842 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882847 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882851 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882856 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882860 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882864 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882869 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882873 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882878 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882883 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882892 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882896 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882900 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882905 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882909 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882913 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882917 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882922 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882926 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882930 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:43:47.884441 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882934 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882939 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882943 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882952 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882956 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882961 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882965 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882970 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882974 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882979 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882983 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882988 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882992 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.882996 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883001 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883008 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883018 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883023 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883028 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:43:47.884977 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883033 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883037 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883043 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883047 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.883052 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884064 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884073 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884078 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884083 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884090 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884100 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884104 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884109 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884114 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884118 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884122 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884127 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884131 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884137 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884141 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:43:47.885446 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884145 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884150 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884154 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884163 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884168 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884171 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884176 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884180 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884184 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884188 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884193 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884198 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884202 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884206 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884211 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884215 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884224 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884267 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884304 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884410 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:43:47.885936 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884416 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884419 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884423 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884426 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884431 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884435 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884438 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884441 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884444 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884448 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884450 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884453 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884456 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884459 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884462 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884465 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884467 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884470 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884472 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884477 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:43:47.886433 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884480 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884483 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884485 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884490 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884493 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884496 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884504 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884507 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884511 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884513 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884517 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884520 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884523 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884525 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884528 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884531 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884534 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884536 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884541 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:43:47.886940 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884544 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884548 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884550 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884565 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884567 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884570 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884574 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884577 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884580 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884583 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884589 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.884591 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885169 2578 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885178 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885185 2578 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885190 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885194 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885197 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885202 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885206 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885209 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 16:43:47.887422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885212 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885216 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885220 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885224 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885227 2578 flags.go:64] FLAG: --cgroup-root="" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885230 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885233 2578 flags.go:64] FLAG: --client-ca-file="" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885235 2578 flags.go:64] FLAG: --cloud-config="" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885238 2578 flags.go:64] FLAG: --cloud-provider="external" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885241 2578 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885245 2578 flags.go:64] FLAG: --cluster-domain="" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885248 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885252 2578 flags.go:64] FLAG: --config-dir="" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885254 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885258 2578 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885262 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885265 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885268 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885271 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885274 2578 flags.go:64] FLAG: --contention-profiling="false" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885277 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885280 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885283 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885286 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885290 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 16:43:47.887960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885293 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885296 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885299 2578 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885303 2578 flags.go:64] FLAG: --enable-server="true" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885306 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885310 2578 flags.go:64] FLAG: --event-burst="100" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885314 2578 flags.go:64] FLAG: --event-qps="50" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885316 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885321 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885324 2578 flags.go:64] FLAG: --eviction-hard="" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885332 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885336 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885339 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885342 2578 flags.go:64] FLAG: --eviction-soft="" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885344 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885347 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885350 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885354 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885357 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885360 2578 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885363 2578 flags.go:64] FLAG: --feature-gates="" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885366 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885369 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885372 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885375 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885378 2578 flags.go:64] FLAG: --healthz-port="10248" Jun 03 16:43:47.888581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885381 2578 flags.go:64] FLAG: --help="false" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885384 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-136-251.ec2.internal" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885387 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885390 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885393 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885396 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885400 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885403 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885405 2578 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885408 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885412 2578 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885415 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885418 2578 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885421 2578 flags.go:64] FLAG: --kube-reserved="" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885424 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885427 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885431 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885434 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885437 2578 flags.go:64] FLAG: --lock-file="" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885439 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885442 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885445 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885451 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 16:43:47.889218 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885454 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885457 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885460 2578 flags.go:64] FLAG: --logging-format="text" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885463 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885466 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885469 2578 flags.go:64] FLAG: --manifest-url="" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885472 2578 flags.go:64] FLAG: --manifest-url-header="" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885477 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885480 2578 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885484 2578 flags.go:64] FLAG: --max-pods="110" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885487 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885490 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885493 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885496 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885499 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885502 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885505 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885513 2578 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885516 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885519 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885523 2578 flags.go:64] FLAG: --pod-cidr="" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885526 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885532 2578 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885535 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 16:43:47.889818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885538 2578 flags.go:64] FLAG: --pods-per-core="0" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885542 2578 flags.go:64] FLAG: --port="10250" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885545 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885548 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0362fdcb8b71108fa" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885551 2578 flags.go:64] FLAG: --qos-reserved="" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885569 2578 flags.go:64] FLAG: --read-only-port="10255" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885572 2578 flags.go:64] FLAG: --register-node="true" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885575 2578 flags.go:64] FLAG: --register-schedulable="true" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885578 2578 flags.go:64] FLAG: --register-with-taints="" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885581 2578 flags.go:64] FLAG: --registry-burst="10" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885584 2578 flags.go:64] FLAG: --registry-qps="5" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885587 2578 flags.go:64] FLAG: --reserved-cpus="" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885590 2578 flags.go:64] FLAG: --reserved-memory="" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885594 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885597 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885600 2578 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885603 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885605 2578 flags.go:64] FLAG: --runonce="false" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885608 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885611 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885614 2578 flags.go:64] FLAG: --seccomp-default="false" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885617 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885619 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885623 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885626 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885628 2578 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 16:43:47.890428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885631 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885634 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885639 2578 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885642 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885645 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885648 2578 flags.go:64] FLAG: --system-cgroups="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885652 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885658 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885661 2578 flags.go:64] FLAG: --tls-cert-file="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885664 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885667 2578 flags.go:64] FLAG: --tls-min-version="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885670 2578 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885673 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885676 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885679 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885681 2578 flags.go:64] FLAG: --v="2" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885686 2578 flags.go:64] FLAG: --version="false" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885690 2578 flags.go:64] FLAG: --vmodule="" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885693 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.885697 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885784 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885788 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885791 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885794 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:43:47.891068 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885797 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885800 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885802 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885805 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885807 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885809 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885812 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885815 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885817 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885820 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885824 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885827 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885830 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885832 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885837 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885839 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885842 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885844 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885847 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885849 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:43:47.891713 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885852 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885854 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885857 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885860 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885862 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885865 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885869 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885873 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885876 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885878 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885881 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885884 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885886 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885889 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885891 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885894 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885896 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885899 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885901 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:43:47.892246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885904 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885906 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885909 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885915 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885917 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885920 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885922 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885926 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885929 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885932 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885934 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885937 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885939 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885942 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885945 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885947 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885950 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885952 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885955 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:43:47.892737 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885958 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885960 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885963 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885966 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885968 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885971 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885974 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885976 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885979 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885981 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885984 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885986 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885989 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885992 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885994 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.885997 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886000 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886003 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886005 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886008 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:43:47.893213 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886012 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886016 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886019 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.886022 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.886720 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.893412 2578 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.893429 2578 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893472 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893477 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893481 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893484 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893487 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893489 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893492 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893495 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:43:47.893721 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893498 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893501 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893503 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893506 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893508 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893511 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893513 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893516 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893519 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893521 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893524 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893527 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893529 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893532 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893534 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893537 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893539 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893542 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893544 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893547 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:43:47.894108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893549 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893552 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893568 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893574 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893577 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893580 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893584 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893588 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893591 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893594 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893597 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893600 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893602 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893605 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893608 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893611 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893613 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893616 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893618 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:43:47.894620 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893621 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893624 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893626 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893629 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893632 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893634 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893637 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893640 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893642 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893645 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893647 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893650 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893652 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893655 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893657 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893660 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893663 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893667 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893670 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893672 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:43:47.895087 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893675 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893677 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893680 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893682 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893685 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893687 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893690 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893692 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893695 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893698 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893700 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893703 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893705 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893708 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893712 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893715 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893717 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893720 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:43:47.895580 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893722 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.893728 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893816 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893820 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893823 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893826 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893829 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893833 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893836 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893839 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893842 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893845 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893849 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893851 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893854 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:43:47.896027 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893856 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893859 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893862 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893865 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893867 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893870 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893872 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893875 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893878 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893880 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893883 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893885 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893888 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893890 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893893 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893896 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893898 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893901 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893903 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893906 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:43:47.896402 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893908 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893911 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893914 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893916 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893919 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893921 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893924 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893927 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893930 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893933 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893936 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893939 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893942 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893945 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893947 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893950 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893952 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893955 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893957 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893960 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:43:47.896914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893962 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893964 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893967 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893969 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893972 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893974 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893977 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893980 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893982 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893985 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893987 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893990 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893992 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893994 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893997 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.893999 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894002 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894004 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894007 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894010 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:43:47.897407 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894013 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894015 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894019 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894023 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894026 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894029 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894032 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894034 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894037 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894040 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894043 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894045 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:47.894048 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.894053 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:43:47.897905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.894648 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 16:43:47.898253 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.898228 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 16:43:47.899129 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.899117 2578 server.go:1019] "Starting client certificate rotation" Jun 03 16:43:47.899230 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.899214 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:43:47.899262 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.899254 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:43:47.918932 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.918912 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:43:47.922055 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.922038 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:43:47.935002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.934984 2578 log.go:25] "Validated CRI v1 runtime API" Jun 03 16:43:47.940112 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.940098 2578 log.go:25] "Validated CRI v1 image API" Jun 03 16:43:47.941286 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.941269 2578 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 16:43:47.943354 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.943337 2578 fs.go:135] Filesystem UUIDs: map[397d9045-dff3-4fcb-9759-e31db541b19e:/dev/nvme0n1p4 64cf2492-2bef-4a70-9ada-605f8285b184:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 03 16:43:47.943417 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.943354 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 16:43:47.947465 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.947365 2578 manager.go:217] Machine: {Timestamp:2026-06-03 16:43:47.946898596 +0000 UTC m=+0.330246026 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099224 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21d5cc1da7100061b8de7980e0d375 SystemUUID:ec21d5cc-1da7-1000-61b8-de7980e0d375 BootID:922e4e04-a8a7-483f-973b-bf685a10e22c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:41:79:3d:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:41:79:3d:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:3e:b6:9b:a1:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 16:43:47.947854 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.947843 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 16:43:47.947931 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.947919 2578 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 16:43:47.949644 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.949631 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:43:47.950423 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.950401 2578 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 16:43:47.950552 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.950426 2578 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-251.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 16:43:47.950613 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.950573 2578 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 16:43:47.950613 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.950582 2578 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 16:43:47.950613 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.950589 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:43:47.951294 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.951283 2578 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:43:47.952128 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.952117 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:43:47.952230 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.952221 2578 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 16:43:47.954205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.954195 2578 kubelet.go:491] "Attempting to sync node with API server" Jun 03 16:43:47.954245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.954208 2578 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 16:43:47.954245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.954220 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 16:43:47.954245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.954229 2578 kubelet.go:397] "Adding apiserver pod source" Jun 03 16:43:47.954245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.954237 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 16:43:47.955209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.955194 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:43:47.955282 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.955214 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:43:47.957655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.957631 2578 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 16:43:47.959372 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.959359 2578 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 16:43:47.961064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961044 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 16:43:47.961064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961067 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961080 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961087 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961096 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961102 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961108 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961113 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961120 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961126 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961145 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 16:43:47.961185 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961151 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 16:43:47.961733 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961723 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 16:43:47.961770 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.961734 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 16:43:47.964750 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.964735 2578 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 16:43:47.964825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.964768 2578 server.go:1295] "Started kubelet" Jun 03 16:43:47.964887 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.964853 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 16:43:47.964938 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.964863 2578 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 16:43:47.964971 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.964963 2578 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 16:43:47.965578 ip-10-0-136-251 systemd[1]: Started Kubernetes Kubelet. Jun 03 16:43:47.966398 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.966284 2578 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 16:43:47.966398 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.966286 2578 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-251.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 03 16:43:47.969535 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:47.969506 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-251.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 16:43:47.969630 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.969580 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9fzsx" Jun 03 16:43:47.969697 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:47.969674 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 16:43:47.971358 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.971335 2578 server.go:317] "Adding debug handlers to kubelet server" Jun 03 16:43:47.973817 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:47.971491 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-251.ec2.internal.18b59fe3b0a2b93f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-251.ec2.internal,UID:ip-10-0-136-251.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-251.ec2.internal,},FirstTimestamp:2026-06-03 16:43:47.964746047 +0000 UTC m=+0.348093456,LastTimestamp:2026-06-03 16:43:47.964746047 +0000 UTC m=+0.348093456,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-251.ec2.internal,}" Jun 03 16:43:47.975094 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:47.975063 2578 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 16:43:47.975195 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975121 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 16:43:47.975573 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975539 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 16:43:47.975656 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975623 2578 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 16:43:47.975656 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975634 2578 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 16:43:47.975746 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975706 2578 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 16:43:47.975817 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975791 2578 reconstruct.go:97] "Volume reconstruction finished" Jun 03 16:43:47.975817 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.975803 2578 reconciler.go:26] "Reconciler: start to sync state" Jun 03 16:43:47.976300 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:47.976282 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:47.976606 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976594 2578 factory.go:153] Registering CRI-O factory Jun 03 16:43:47.976659 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976611 2578 factory.go:223] Registration of the crio container factory successfully Jun 03 16:43:47.976716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976663 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 16:43:47.976716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976674 2578 factory.go:55] Registering systemd factory Jun 03 16:43:47.976716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976680 2578 factory.go:223] Registration of the systemd container factory successfully Jun 03 16:43:47.976716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976694 2578 factory.go:103] Registering Raw factory Jun 03 16:43:47.976716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.976713 2578 manager.go:1196] Started watching for new ooms in manager Jun 03 16:43:47.977101 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.977089 2578 manager.go:319] Starting recovery of all containers Jun 03 16:43:47.980602 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.980579 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9fzsx" Jun 03 16:43:47.986491 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.986388 2578 manager.go:324] Recovery completed Jun 03 16:43:47.990170 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.990155 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:43:47.990900 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.990879 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:43:47.992086 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.992073 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:43:47.992144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.992096 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:43:47.992144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.992107 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:43:47.992519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.992507 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 16:43:47.992519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.992519 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 16:43:47.992624 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.992534 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:43:47.993738 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:47.993724 2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-251.ec2.internal\" not found" node="ip-10-0-136-251.ec2.internal" Jun 03 16:43:47.994675 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.994664 2578 policy_none.go:49] "None policy: Start" Jun 03 16:43:47.994714 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.994679 2578 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 16:43:47.994714 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:47.994689 2578 state_mem.go:35] "Initializing new in-memory state store" Jun 03 16:43:48.031131 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031117 2578 manager.go:341] "Starting Device Plugin manager" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.031145 2578 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031154 2578 server.go:85] "Starting device plugin registration server" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031370 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031382 2578 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031471 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031549 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.031577 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.032086 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 16:43:48.039379 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.032121 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.093018 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.092993 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 16:43:48.094311 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.094291 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 16:43:48.094311 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.094311 2578 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 16:43:48.094448 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.094325 2578 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 16:43:48.094448 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.094332 2578 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 16:43:48.094448 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.094365 2578 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 16:43:48.097576 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.097540 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:43:48.131655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.131612 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:43:48.132386 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.132360 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:43:48.132463 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.132393 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:43:48.132463 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.132403 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:43:48.132463 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.132423 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.141176 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.141160 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.141222 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.141179 2578 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-251.ec2.internal\": node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.158975 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.158956 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.194671 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.194653 2578 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal"] Jun 03 16:43:48.194726 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.194702 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:43:48.195906 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.195892 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:43:48.195985 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.195917 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:43:48.195985 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.195935 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:43:48.196947 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.196935 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:43:48.197075 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197062 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.197129 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197089 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:43:48.197611 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197595 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:43:48.197699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197620 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:43:48.197699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197627 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:43:48.197699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197647 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:43:48.197699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197630 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:43:48.197699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.197661 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:43:48.198688 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.198674 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.198748 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.198698 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:43:48.199319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.199305 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:43:48.199392 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.199326 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:43:48.199392 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.199339 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:43:48.221221 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.221201 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-251.ec2.internal\" not found" node="ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.225435 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.225419 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-251.ec2.internal\" not found" node="ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.259379 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.259361 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.277727 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.277707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77d77f19d5202da2dce27948f5d2589d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-251.ec2.internal\" (UID: \"77d77f19d5202da2dce27948f5d2589d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.277784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.277734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e5a9010e436254c451d3c9f438516342-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal\" (UID: \"e5a9010e436254c451d3c9f438516342\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.277784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.277752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a9010e436254c451d3c9f438516342-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal\" (UID: \"e5a9010e436254c451d3c9f438516342\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.359463 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.359445 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.378776 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.378758 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77d77f19d5202da2dce27948f5d2589d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-251.ec2.internal\" (UID: \"77d77f19d5202da2dce27948f5d2589d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.378829 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.378783 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e5a9010e436254c451d3c9f438516342-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal\" (UID: \"e5a9010e436254c451d3c9f438516342\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.378829 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.378800 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a9010e436254c451d3c9f438516342-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal\" (UID: \"e5a9010e436254c451d3c9f438516342\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.378890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.378835 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a9010e436254c451d3c9f438516342-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal\" (UID: \"e5a9010e436254c451d3c9f438516342\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.378890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.378859 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77d77f19d5202da2dce27948f5d2589d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-251.ec2.internal\" (UID: \"77d77f19d5202da2dce27948f5d2589d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.378890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.378864 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e5a9010e436254c451d3c9f438516342-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal\" (UID: \"e5a9010e436254c451d3c9f438516342\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.460206 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.460146 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.524567 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.524535 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.528972 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.528953 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:48.560542 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.560520 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.661012 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.660978 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.761479 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.761429 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.861957 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.861926 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.899415 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.899392 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 16:43:48.899938 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.899516 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:43:48.899938 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.899544 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:43:48.962978 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:48.962953 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:48.975342 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.975319 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 16:43:48.982116 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.982088 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 16:38:47 +0000 UTC" deadline="2028-03-09 01:53:00.04442217 +0000 UTC" Jun 03 16:43:48.982116 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.982114 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15465h9m11.062311862s" Jun 03 16:43:48.985287 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:48.985270 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:43:49.006538 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.006518 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jpzdq" Jun 03 16:43:49.014591 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:49.014531 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a9010e436254c451d3c9f438516342.slice/crio-deb492d0da20f87825502e6a2123ab8d19447f7b1f4d92bb5f600f7043b04f6c WatchSource:0}: Error finding container deb492d0da20f87825502e6a2123ab8d19447f7b1f4d92bb5f600f7043b04f6c: Status 404 returned error can't find the container with id deb492d0da20f87825502e6a2123ab8d19447f7b1f4d92bb5f600f7043b04f6c Jun 03 16:43:49.014755 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:49.014735 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d77f19d5202da2dce27948f5d2589d.slice/crio-02482317b784019d46cba6dd1a4dbbe5f68777ec946bf0694f03694684483ace WatchSource:0}: Error finding container 02482317b784019d46cba6dd1a4dbbe5f68777ec946bf0694f03694684483ace: Status 404 returned error can't find the container with id 02482317b784019d46cba6dd1a4dbbe5f68777ec946bf0694f03694684483ace Jun 03 16:43:49.015877 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.015858 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jpzdq" Jun 03 16:43:49.018246 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.018231 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:43:49.029159 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.029141 2578 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:43:49.063613 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:49.063578 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:49.097427 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.097385 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" event={"ID":"e5a9010e436254c451d3c9f438516342","Type":"ContainerStarted","Data":"deb492d0da20f87825502e6a2123ab8d19447f7b1f4d92bb5f600f7043b04f6c"} Jun 03 16:43:49.098265 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.098245 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" event={"ID":"77d77f19d5202da2dce27948f5d2589d","Type":"ContainerStarted","Data":"02482317b784019d46cba6dd1a4dbbe5f68777ec946bf0694f03694684483ace"} Jun 03 16:43:49.164433 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:49.164402 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:49.264958 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:49.264899 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-251.ec2.internal\" not found" Jun 03 16:43:49.321092 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.321067 2578 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:43:49.376626 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.376606 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" Jun 03 16:43:49.387192 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.387168 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:43:49.388043 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.388028 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" Jun 03 16:43:49.397528 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.397510 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:43:49.955018 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.954991 2578 apiserver.go:52] "Watching apiserver" Jun 03 16:43:49.963617 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.963597 2578 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 16:43:49.965131 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.965100 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-lp8cj","kube-system/konnectivity-agent-ht9gl","kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n","openshift-dns/node-resolver-xmbrr","openshift-network-diagnostics/network-check-target-9sggr","openshift-network-operator/iptables-alerter-zsc6l","openshift-cluster-node-tuning-operator/tuned-ql6tk","openshift-image-registry/node-ca-l74wk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal","openshift-multus/multus-4nbf5","openshift-multus/multus-additional-cni-plugins-8qqp5","openshift-multus/network-metrics-daemon-rn64j"] Jun 03 16:43:49.966594 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.966575 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:49.967669 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.967648 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:49.968905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.968483 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.968905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.968522 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.968905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.968823 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 16:43:49.968905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.968849 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lzxm7\"" Jun 03 16:43:49.969372 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.969342 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 16:43:49.969516 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.969455 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 16:43:49.969650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.969599 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ctrdt\"" Jun 03 16:43:49.970637 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.969925 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.971269 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.970912 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:49.971269 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.971003 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:49.971269 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:49.971074 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:43:49.971654 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.971582 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.971818 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.971802 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ktf7c\"" Jun 03 16:43:49.971892 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.971804 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.971892 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.971847 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 16:43:49.972620 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.972258 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.972620 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.972272 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.972824 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.972802 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.973214 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.973196 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r2qzg\"" Jun 03 16:43:49.973848 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.973530 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.973937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.973917 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 16:43:49.973998 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.973987 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 16:43:49.974051 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.974006 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 16:43:49.974431 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.974206 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ssw54\"" Jun 03 16:43:49.974431 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.974255 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.974431 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.974315 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 16:43:49.974431 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.974380 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.974996 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.974978 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:49.975109 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.975093 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.975173 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.975142 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wfb7h\"" Jun 03 16:43:49.975379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.975363 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.976580 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.976546 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.977470 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.976829 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.977470 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.976904 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 16:43:49.977470 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.976833 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rptgk\"" Jun 03 16:43:49.977673 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.977519 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.978165 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.978146 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 16:43:49.978425 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.978409 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 16:43:49.978509 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.978438 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:49.979155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.978795 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 16:43:49.979155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.978815 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 16:43:49.979155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.978918 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vt5lj\"" Jun 03 16:43:49.980485 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.980293 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:49.980485 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:49.980366 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:43:49.980485 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.980384 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 16:43:49.980485 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.980409 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xctd4\"" Jun 03 16:43:49.981326 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.981301 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 16:43:49.985949 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.985924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-cni-netd\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.986043 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.985981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.986043 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-ovnkube-config\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.986155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5w22\" (UniqueName: \"kubernetes.io/projected/b54328d7-952f-4cf4-a355-b6a2a697354c-kube-api-access-l5w22\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.986155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-registration-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.986877 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-lib-modules\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.986963 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986917 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-k8s-cni-cncf-io\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.986963 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-kubelet\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987071 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.986983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-systemd\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987071 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-var-lib-kubelet\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987071 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-etc-selinux\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.987207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987098 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-sys-fs\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.987207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-hostroot\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-daemon-config\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-kubernetes\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-cni-multus\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-multus-certs\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-modprobe-d\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-run\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-sys\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-host\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5gdn\" (UniqueName: \"kubernetes.io/projected/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-kube-api-access-p5gdn\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987376 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06cc3bd3-c291-4ad8-8515-bb30440a138f-host\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:49.987400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-run-netns\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysctl-conf\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-os-release\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/142582a7-24a0-4b02-bcba-a4554f275c8e-agent-certs\") pod \"konnectivity-agent-ht9gl\" (UID: \"142582a7-24a0-4b02-bcba-a4554f275c8e\") " pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-etc-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-systemd\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-system-cni-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-systemd-units\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-ovn\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-cni-bin\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-hosts-file\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-tuned\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-log-socket\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-tmp-dir\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:49.987879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-cnibin\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987901 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-cni-binary-copy\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rgln\" (UniqueName: \"kubernetes.io/projected/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-kube-api-access-9rgln\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.987992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-node-log\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkw5m\" (UniqueName: \"kubernetes.io/projected/d4364c3c-6642-4620-9776-3dc36f3526aa-kube-api-access-hkw5m\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-cni-bin\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-conf-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-ovnkube-script-lib\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06cc3bd3-c291-4ad8-8515-bb30440a138f-serviceca\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-tmp\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/142582a7-24a0-4b02-bcba-a4554f275c8e-konnectivity-ca\") pod \"konnectivity-agent-ht9gl\" (UID: \"142582a7-24a0-4b02-bcba-a4554f275c8e\") " pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-slash\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-var-lib-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2px8c\" (UniqueName: \"kubernetes.io/projected/06cc3bd3-c291-4ad8-8515-bb30440a138f-kube-api-access-2px8c\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-kubelet\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4364c3c-6642-4620-9776-3dc36f3526aa-iptables-alerter-script\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:49.988601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-device-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysconfig\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988537 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-netns\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4364c3c-6642-4620-9776-3dc36f3526aa-host-slash\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysctl-d\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-cni-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-run-ovn-kubernetes\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-env-overrides\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-socket-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988779 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8nrd\" (UniqueName: \"kubernetes.io/projected/ed72f769-6bc9-433b-9fb4-58960af9908a-kube-api-access-h8nrd\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txttn\" (UniqueName: \"kubernetes.io/projected/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-kube-api-access-txttn\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-socket-dir-parent\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b54328d7-952f-4cf4-a355-b6a2a697354c-ovn-node-metrics-cert\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:49.989319 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:49.988948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-etc-kubernetes\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.016475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.016444 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:38:49 +0000 UTC" deadline="2027-11-25 02:56:29.776143842 +0000 UTC" Jun 03 16:43:50.016475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.016473 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12946h12m39.759673784s" Jun 03 16:43:50.076778 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.076756 2578 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 16:43:50.089632 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089601 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-ovnkube-script-lib\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.089739 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089644 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06cc3bd3-c291-4ad8-8515-bb30440a138f-serviceca\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.089739 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089671 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-tmp\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.089837 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089784 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/142582a7-24a0-4b02-bcba-a4554f275c8e-konnectivity-ca\") pod \"konnectivity-agent-ht9gl\" (UID: \"142582a7-24a0-4b02-bcba-a4554f275c8e\") " pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:50.089837 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089811 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-slash\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.089837 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089827 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-var-lib-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.089976 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089843 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2px8c\" (UniqueName: \"kubernetes.io/projected/06cc3bd3-c291-4ad8-8515-bb30440a138f-kube-api-access-2px8c\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.089976 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089884 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-var-lib-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.089976 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089897 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-slash\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.089976 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.089947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cni-binary-copy\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090157 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-kubelet\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090324 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090368 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4364c3c-6642-4620-9776-3dc36f3526aa-iptables-alerter-script\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090406 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-device-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090439 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysconfig\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090471 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-netns\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090502 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4364c3c-6642-4620-9776-3dc36f3526aa-host-slash\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090528 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090575 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysctl-d\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090574 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-device-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090553 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06cc3bd3-c291-4ad8-8515-bb30440a138f-serviceca\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090628 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-cni-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090672 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysconfig\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090668 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-run-ovn-kubernetes\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090724 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-run-ovn-kubernetes\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-netns\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090760 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-env-overrides\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.093952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4364c3c-6642-4620-9776-3dc36f3526aa-host-slash\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090795 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-socket-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090829 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h8nrd\" (UniqueName: \"kubernetes.io/projected/ed72f769-6bc9-433b-9fb4-58960af9908a-kube-api-access-h8nrd\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090834 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-kubelet\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/142582a7-24a0-4b02-bcba-a4554f275c8e-konnectivity-ca\") pod \"konnectivity-agent-ht9gl\" (UID: \"142582a7-24a0-4b02-bcba-a4554f275c8e\") " pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090940 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-cni-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090973 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysctl-d\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091011 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-socket-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091160 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-txttn\" (UniqueName: \"kubernetes.io/projected/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-kube-api-access-txttn\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091207 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091262 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-socket-dir-parent\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091294 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b54328d7-952f-4cf4-a355-b6a2a697354c-ovn-node-metrics-cert\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-os-release\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091365 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-etc-kubernetes\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091397 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-cni-netd\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091400 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-env-overrides\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.094866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091425 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091458 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-ovnkube-config\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091489 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l5w22\" (UniqueName: \"kubernetes.io/projected/b54328d7-952f-4cf4-a355-b6a2a697354c-kube-api-access-l5w22\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091492 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4364c3c-6642-4620-9776-3dc36f3526aa-iptables-alerter-script\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091521 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-registration-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091569 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-lib-modules\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091611 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-etc-kubernetes\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091657 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-k8s-cni-cncf-io\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091693 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-lib-modules\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-k8s-cni-cncf-io\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091745 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-kubelet\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091806 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091816 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-registration-dir\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092053 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-cni-netd\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.091695 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-kubelet\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092103 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-systemd\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h7qt\" (UniqueName: \"kubernetes.io/projected/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-kube-api-access-2h7qt\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.095650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092169 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-var-lib-kubelet\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092195 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-etc-selinux\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092226 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-sys-fs\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092255 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-hostroot\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092288 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-daemon-config\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092310 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-ovnkube-config\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092321 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-kubernetes\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092351 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-cni-multus\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092389 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-multus-certs\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092445 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-etc-selinux\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092457 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-modprobe-d\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092485 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-run\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-systemd\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092537 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-sys\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092577 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-host\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092601 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-var-lib-kubelet\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.096339 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092607 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5gdn\" (UniqueName: \"kubernetes.io/projected/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-kube-api-access-p5gdn\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092638 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06cc3bd3-c291-4ad8-8515-bb30440a138f-host\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092656 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-run-multus-certs\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cnibin\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092723 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed72f769-6bc9-433b-9fb4-58960af9908a-sys-fs\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092756 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-run-netns\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092792 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysctl-conf\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092793 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-modprobe-d\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-run\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092859 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-hostroot\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-sys\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092925 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-kubernetes\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092942 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-os-release\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.092975 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/142582a7-24a0-4b02-bcba-a4554f275c8e-agent-certs\") pod \"konnectivity-agent-ht9gl\" (UID: \"142582a7-24a0-4b02-bcba-a4554f275c8e\") " pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093006 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-etc-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093035 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-systemd\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093061 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-system-cni-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093090 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-systemd-units\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.097120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093119 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093150 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-ovn\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093201 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-ovn\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093220 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-socket-dir-parent\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093229 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-cni-multus\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093242 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-run-netns\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093269 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-cni-bin\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093303 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4l82\" (UniqueName: \"kubernetes.io/projected/e0cf9100-e9b8-42a0-93a8-f273610a87cf-kube-api-access-w4l82\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093364 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-sysctl-conf\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093369 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-hosts-file\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093401 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-tuned\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093432 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-host\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093437 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-system-cni-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093476 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-host-cni-bin\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-log-socket\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-etc-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-systemd\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.098002 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093701 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-os-release\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093759 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06cc3bd3-c291-4ad8-8515-bb30440a138f-host\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093902 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-systemd-units\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093954 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-run-openvswitch\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-hosts-file\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.093434 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-log-socket\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094251 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094294 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-tmp-dir\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094404 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-daemon-config\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094421 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b54328d7-952f-4cf4-a355-b6a2a697354c-ovn-node-metrics-cert\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094465 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-cnibin\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094519 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-cni-binary-copy\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094550 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9rgln\" (UniqueName: \"kubernetes.io/projected/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-kube-api-access-9rgln\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-system-cni-dir\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094626 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-node-log\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094694 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hkw5m\" (UniqueName: \"kubernetes.io/projected/d4364c3c-6642-4620-9776-3dc36f3526aa-kube-api-access-hkw5m\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094756 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-cni-bin\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.098916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094788 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-conf-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.094874 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-multus-conf-dir\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.095127 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b54328d7-952f-4cf4-a355-b6a2a697354c-node-log\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.095136 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-cni-binary-copy\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.095217 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-host-var-lib-cni-bin\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.095412 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-tmp\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.095465 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-tmp-dir\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.095518 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-cnibin\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.090732 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b54328d7-952f-4cf4-a355-b6a2a697354c-ovnkube-script-lib\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.097990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/142582a7-24a0-4b02-bcba-a4554f275c8e-agent-certs\") pod \"konnectivity-agent-ht9gl\" (UID: \"142582a7-24a0-4b02-bcba-a4554f275c8e\") " pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.098035 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-etc-tuned\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.098507 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.098525 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.098539 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.098622 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:43:50.59860179 +0000 UTC m=+2.981949205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:50.099980 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.099777 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2px8c\" (UniqueName: \"kubernetes.io/projected/06cc3bd3-c291-4ad8-8515-bb30440a138f-kube-api-access-2px8c\") pod \"node-ca-l74wk\" (UID: \"06cc3bd3-c291-4ad8-8515-bb30440a138f\") " pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.100920 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.100894 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8nrd\" (UniqueName: \"kubernetes.io/projected/ed72f769-6bc9-433b-9fb4-58960af9908a-kube-api-access-h8nrd\") pod \"aws-ebs-csi-driver-node-x8c5n\" (UID: \"ed72f769-6bc9-433b-9fb4-58960af9908a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.100999 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.100914 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5w22\" (UniqueName: \"kubernetes.io/projected/b54328d7-952f-4cf4-a355-b6a2a697354c-kube-api-access-l5w22\") pod \"ovnkube-node-lp8cj\" (UID: \"b54328d7-952f-4cf4-a355-b6a2a697354c\") " pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.101939 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.101914 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txttn\" (UniqueName: \"kubernetes.io/projected/fa1a1e82-367d-49e7-9abb-6f49ef988e5d-kube-api-access-txttn\") pod \"node-resolver-xmbrr\" (UID: \"fa1a1e82-367d-49e7-9abb-6f49ef988e5d\") " pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.102312 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.102288 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5gdn\" (UniqueName: \"kubernetes.io/projected/da2535e6-b0fc-42d5-8f6f-5a9b1781eff4-kube-api-access-p5gdn\") pod \"tuned-ql6tk\" (UID: \"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4\") " pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.102993 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.102948 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rgln\" (UniqueName: \"kubernetes.io/projected/feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4-kube-api-access-9rgln\") pod \"multus-4nbf5\" (UID: \"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4\") " pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.103233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.103212 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkw5m\" (UniqueName: \"kubernetes.io/projected/d4364c3c-6642-4620-9776-3dc36f3526aa-kube-api-access-hkw5m\") pod \"iptables-alerter-zsc6l\" (UID: \"d4364c3c-6642-4620-9776-3dc36f3526aa\") " pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.152028 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.151999 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:43:50.196061 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196034 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196077 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cnibin\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196112 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4l82\" (UniqueName: \"kubernetes.io/projected/e0cf9100-e9b8-42a0-93a8-f273610a87cf-kube-api-access-w4l82\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:50.196207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196138 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196166 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196447 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196447 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196354 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-system-cni-dir\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196447 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196389 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:50.196447 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196425 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cni-binary-copy\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196658 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196474 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-os-release\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196658 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196509 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2h7qt\" (UniqueName: \"kubernetes.io/projected/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-kube-api-access-2h7qt\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196658 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196639 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196841 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.196746 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:50.196841 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.196800 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:43:50.696786627 +0000 UTC m=+3.080134025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:50.196944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196886 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-system-cni-dir\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196918 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cnibin\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.196944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.196929 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-os-release\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.197174 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.197156 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cni-binary-copy\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.197756 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.197731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.209184 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.209132 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h7qt\" (UniqueName: \"kubernetes.io/projected/4f5d7b02-bc02-4e14-8aa3-5b489f48826c-kube-api-access-2h7qt\") pod \"multus-additional-cni-plugins-8qqp5\" (UID: \"4f5d7b02-bc02-4e14-8aa3-5b489f48826c\") " pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.209184 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.209133 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4l82\" (UniqueName: \"kubernetes.io/projected/e0cf9100-e9b8-42a0-93a8-f273610a87cf-kube-api-access-w4l82\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:50.281025 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.280994 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zsc6l" Jun 03 16:43:50.291751 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.291727 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:43:50.301483 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.301464 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" Jun 03 16:43:50.307025 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.307008 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xmbrr" Jun 03 16:43:50.312538 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.312524 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:43:50.318086 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.318064 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" Jun 03 16:43:50.324640 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.324622 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l74wk" Jun 03 16:43:50.331152 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.331133 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4nbf5" Jun 03 16:43:50.335597 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.335579 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" Jun 03 16:43:50.384296 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.384278 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:43:50.589542 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.589513 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4364c3c_6642_4620_9776_3dc36f3526aa.slice/crio-ea4ef76c5727e2ef86addd4bcfe7dce9363eb5d4a1db13cfb89e1fd50430d3ed WatchSource:0}: Error finding container ea4ef76c5727e2ef86addd4bcfe7dce9363eb5d4a1db13cfb89e1fd50430d3ed: Status 404 returned error can't find the container with id ea4ef76c5727e2ef86addd4bcfe7dce9363eb5d4a1db13cfb89e1fd50430d3ed Jun 03 16:43:50.590358 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.590312 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2535e6_b0fc_42d5_8f6f_5a9b1781eff4.slice/crio-45837e61a08784999553b605f694017b93d5171a5795c76dfd7a84b762925ee3 WatchSource:0}: Error finding container 45837e61a08784999553b605f694017b93d5171a5795c76dfd7a84b762925ee3: Status 404 returned error can't find the container with id 45837e61a08784999553b605f694017b93d5171a5795c76dfd7a84b762925ee3 Jun 03 16:43:50.592400 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.591800 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5d7b02_bc02_4e14_8aa3_5b489f48826c.slice/crio-b88c81fb8b8a50ffc20396448e23ca7fa717034e28c3a04f337860fa0e3057d4 WatchSource:0}: Error finding container b88c81fb8b8a50ffc20396448e23ca7fa717034e28c3a04f337860fa0e3057d4: Status 404 returned error can't find the container with id b88c81fb8b8a50ffc20396448e23ca7fa717034e28c3a04f337860fa0e3057d4 Jun 03 16:43:50.594632 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.594612 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded72f769_6bc9_433b_9fb4_58960af9908a.slice/crio-b7c0a7827d4dc07c01acedee596defa98d06fed36d5ebda21835d8e73bd5856b WatchSource:0}: Error finding container b7c0a7827d4dc07c01acedee596defa98d06fed36d5ebda21835d8e73bd5856b: Status 404 returned error can't find the container with id b7c0a7827d4dc07c01acedee596defa98d06fed36d5ebda21835d8e73bd5856b Jun 03 16:43:50.595702 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.595648 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142582a7_24a0_4b02_bcba_a4554f275c8e.slice/crio-ca41a584f3bdac638400ae27847e15270c7620881a1a03da6a75f336e848c12a WatchSource:0}: Error finding container ca41a584f3bdac638400ae27847e15270c7620881a1a03da6a75f336e848c12a: Status 404 returned error can't find the container with id ca41a584f3bdac638400ae27847e15270c7620881a1a03da6a75f336e848c12a Jun 03 16:43:50.597024 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.596994 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1a1e82_367d_49e7_9abb_6f49ef988e5d.slice/crio-621979ade22b6e62bff94bc5ce449774d5f4b416035e0cc733fa07f3fe0cefa5 WatchSource:0}: Error finding container 621979ade22b6e62bff94bc5ce449774d5f4b416035e0cc733fa07f3fe0cefa5: Status 404 returned error can't find the container with id 621979ade22b6e62bff94bc5ce449774d5f4b416035e0cc733fa07f3fe0cefa5 Jun 03 16:43:50.598168 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.598107 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06cc3bd3_c291_4ad8_8515_bb30440a138f.slice/crio-040ac8be070aaba21c45055397f416ef3b05880a9cee970074b9eaa51e8d173d WatchSource:0}: Error finding container 040ac8be070aaba21c45055397f416ef3b05880a9cee970074b9eaa51e8d173d: Status 404 returned error can't find the container with id 040ac8be070aaba21c45055397f416ef3b05880a9cee970074b9eaa51e8d173d Jun 03 16:43:50.599181 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.599162 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:50.599296 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.599282 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:43:50.599359 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.599303 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:43:50.599359 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.599314 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:50.599441 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.599364 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:43:51.599345101 +0000 UTC m=+3.982692507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:50.602630 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.602605 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeea66f6_c2cf_43e6_ac4e_6dd8baeec6e4.slice/crio-9b5e279ab04dc0490b4a18370eb37962b55ea0cf4a7a7636d3c64473454a4140 WatchSource:0}: Error finding container 9b5e279ab04dc0490b4a18370eb37962b55ea0cf4a7a7636d3c64473454a4140: Status 404 returned error can't find the container with id 9b5e279ab04dc0490b4a18370eb37962b55ea0cf4a7a7636d3c64473454a4140 Jun 03 16:43:50.603357 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:43:50.603338 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54328d7_952f_4cf4_a355_b6a2a697354c.slice/crio-ffa1c74ac07d9b8a120ba7a82ae2604e7499ad669c9c20934d1f8614cf686e78 WatchSource:0}: Error finding container ffa1c74ac07d9b8a120ba7a82ae2604e7499ad669c9c20934d1f8614cf686e78: Status 404 returned error can't find the container with id ffa1c74ac07d9b8a120ba7a82ae2604e7499ad669c9c20934d1f8614cf686e78 Jun 03 16:43:50.700304 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:50.700282 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:50.700431 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.700415 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:50.700476 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:50.700468 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:43:51.700453203 +0000 UTC m=+4.083800604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:51.017028 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.016958 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:38:49 +0000 UTC" deadline="2027-12-09 11:24:25.704957989 +0000 UTC" Jun 03 16:43:51.017028 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.016997 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13290h40m34.687964867s" Jun 03 16:43:51.116449 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.116326 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ht9gl" event={"ID":"142582a7-24a0-4b02-bcba-a4554f275c8e","Type":"ContainerStarted","Data":"ca41a584f3bdac638400ae27847e15270c7620881a1a03da6a75f336e848c12a"} Jun 03 16:43:51.134118 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.134058 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" event={"ID":"ed72f769-6bc9-433b-9fb4-58960af9908a","Type":"ContainerStarted","Data":"b7c0a7827d4dc07c01acedee596defa98d06fed36d5ebda21835d8e73bd5856b"} Jun 03 16:43:51.167932 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.167892 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" event={"ID":"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4","Type":"ContainerStarted","Data":"45837e61a08784999553b605f694017b93d5171a5795c76dfd7a84b762925ee3"} Jun 03 16:43:51.172851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.172823 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l74wk" event={"ID":"06cc3bd3-c291-4ad8-8515-bb30440a138f","Type":"ContainerStarted","Data":"040ac8be070aaba21c45055397f416ef3b05880a9cee970074b9eaa51e8d173d"} Jun 03 16:43:51.180659 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.180633 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerStarted","Data":"b88c81fb8b8a50ffc20396448e23ca7fa717034e28c3a04f337860fa0e3057d4"} Jun 03 16:43:51.181858 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.181834 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zsc6l" event={"ID":"d4364c3c-6642-4620-9776-3dc36f3526aa","Type":"ContainerStarted","Data":"ea4ef76c5727e2ef86addd4bcfe7dce9363eb5d4a1db13cfb89e1fd50430d3ed"} Jun 03 16:43:51.197497 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.197471 2578 generic.go:358] "Generic (PLEG): container finished" podID="e5a9010e436254c451d3c9f438516342" containerID="5af06ef159942ade63a1a4bae6e1ad1ce6119c8296c27f47413519d260b19dee" exitCode=0 Jun 03 16:43:51.197602 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.197542 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" event={"ID":"e5a9010e436254c451d3c9f438516342","Type":"ContainerDied","Data":"5af06ef159942ade63a1a4bae6e1ad1ce6119c8296c27f47413519d260b19dee"} Jun 03 16:43:51.215398 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.215079 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" event={"ID":"77d77f19d5202da2dce27948f5d2589d","Type":"ContainerStarted","Data":"81f0d0dccf392eb3ca09e2c13a4bd3150300b20c6a8c916f20fda6b02e98a83c"} Jun 03 16:43:51.222904 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.222841 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"ffa1c74ac07d9b8a120ba7a82ae2604e7499ad669c9c20934d1f8614cf686e78"} Jun 03 16:43:51.253212 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.253151 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4nbf5" event={"ID":"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4","Type":"ContainerStarted","Data":"9b5e279ab04dc0490b4a18370eb37962b55ea0cf4a7a7636d3c64473454a4140"} Jun 03 16:43:51.256041 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.255996 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xmbrr" event={"ID":"fa1a1e82-367d-49e7-9abb-6f49ef988e5d","Type":"ContainerStarted","Data":"621979ade22b6e62bff94bc5ce449774d5f4b416035e0cc733fa07f3fe0cefa5"} Jun 03 16:43:51.608159 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.607546 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:51.608159 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:51.607713 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:43:51.608159 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:51.607734 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:43:51.608159 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:51.607747 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:51.608159 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:51.607803 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:43:53.60778441 +0000 UTC m=+5.991131820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:51.708522 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:51.708443 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:51.708699 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:51.708616 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:51.708699 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:51.708679 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:43:53.70865891 +0000 UTC m=+6.092006319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:52.095755 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:52.095688 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:52.096172 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:52.095802 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:43:52.098198 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:52.098175 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:52.098321 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:52.098286 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:43:52.272243 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:52.272207 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" event={"ID":"e5a9010e436254c451d3c9f438516342","Type":"ContainerStarted","Data":"077cf375854aa30210b70acf43cf6f99689210aa892a412c2cfac28f050adbc3"} Jun 03 16:43:52.286090 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:52.285241 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-251.ec2.internal" podStartSLOduration=3.285224784 podStartE2EDuration="3.285224784s" podCreationTimestamp="2026-06-03 16:43:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:43:51.236828259 +0000 UTC m=+3.620175679" watchObservedRunningTime="2026-06-03 16:43:52.285224784 +0000 UTC m=+4.668572208" Jun 03 16:43:53.623193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:53.623123 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:53.623696 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:53.623305 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:43:53.623696 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:53.623330 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:43:53.623696 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:53.623344 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:53.623696 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:53.623409 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:43:57.623390522 +0000 UTC m=+10.006737924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:53.723907 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:53.723870 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:53.724081 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:53.724054 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:53.724170 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:53.724117 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:43:57.724098836 +0000 UTC m=+10.107446241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:54.097937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:54.097003 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:54.097937 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:54.097134 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:43:54.097937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:54.097543 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:54.097937 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:54.097658 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:43:56.095169 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:56.095121 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:56.095646 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:56.095239 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:43:56.095796 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:56.095772 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:56.095920 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:56.095899 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:43:57.659749 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:57.659688 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:57.660193 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:57.659816 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:43:57.660193 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:57.659837 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:43:57.660193 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:57.659851 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:57.660193 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:57.659920 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:44:05.659900399 +0000 UTC m=+18.043247809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:43:57.760912 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:57.760878 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:57.761066 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:57.761031 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:57.761147 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:57.761113 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:44:05.761090898 +0000 UTC m=+18.144438299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:43:58.096590 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:58.096088 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:43:58.096590 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:58.096196 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:43:58.096590 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:43:58.096279 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:43:58.096590 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:43:58.096404 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:00.095015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:00.094972 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:00.095483 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:00.094977 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:00.095483 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:00.095098 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:00.095483 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:00.095204 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:02.094698 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:02.094662 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:02.095158 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:02.094770 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:02.095158 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:02.094846 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:02.095158 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:02.094943 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:04.094594 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:04.094543 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:04.095091 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:04.094682 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:04.095091 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:04.094751 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:04.095091 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:04.094861 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:05.716627 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:05.716589 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:05.717120 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:05.716738 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:44:05.717120 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:05.716754 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:44:05.717120 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:05.716764 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:44:05.717120 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:05.716819 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:44:21.716801947 +0000 UTC m=+34.100149351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:44:05.817018 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:05.816980 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:05.817184 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:05.817153 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:44:05.817240 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:05.817229 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:44:21.817208212 +0000 UTC m=+34.200555614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:44:06.095519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:06.095442 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:06.095697 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:06.095570 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:06.095697 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:06.095614 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:06.095804 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:06.095733 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:08.096449 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:08.096407 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:08.096902 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:08.096526 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:08.096902 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:08.096610 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:08.096902 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:08.096724 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:09.309384 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.309175 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4nbf5" event={"ID":"feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4","Type":"ContainerStarted","Data":"0a29fea2f87d588e83c6924e66ddfad4267954fec55435416a3fc3901c726d58"} Jun 03 16:44:09.310481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.310453 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xmbrr" event={"ID":"fa1a1e82-367d-49e7-9abb-6f49ef988e5d","Type":"ContainerStarted","Data":"f967cd205bbd886a7c5076703a8c94403d6433a7a8a5f5cfdeca9465d6520df3"} Jun 03 16:44:09.311952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.311932 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ht9gl" event={"ID":"142582a7-24a0-4b02-bcba-a4554f275c8e","Type":"ContainerStarted","Data":"f66e637e763ead51dd03675b90cfadc2638e4846da71d73a06551692ab568570"} Jun 03 16:44:09.313624 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.313600 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" event={"ID":"ed72f769-6bc9-433b-9fb4-58960af9908a","Type":"ContainerStarted","Data":"bc7b8a740fc8f255d49521a8cf4a0d92e1a1888bf4ff733187feb5272a164725"} Jun 03 16:44:09.316432 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.316206 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" event={"ID":"da2535e6-b0fc-42d5-8f6f-5a9b1781eff4","Type":"ContainerStarted","Data":"e8e5643b2216b92f67e1654bdeca22b8fd4ab350f31e0b4fe4c67454626a8981"} Jun 03 16:44:09.318221 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.318197 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l74wk" event={"ID":"06cc3bd3-c291-4ad8-8515-bb30440a138f","Type":"ContainerStarted","Data":"09bbe3fdf7301cc023c6ca77abe17e4ef0d0b689627f2eaff097512219bda788"} Jun 03 16:44:09.320206 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.320183 2578 generic.go:358] "Generic (PLEG): container finished" podID="4f5d7b02-bc02-4e14-8aa3-5b489f48826c" containerID="ba8b219df151e62efdfc44418293038bd90c6091fb70ce8cb57784c8961598d4" exitCode=0 Jun 03 16:44:09.320298 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.320263 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerDied","Data":"ba8b219df151e62efdfc44418293038bd90c6091fb70ce8cb57784c8961598d4"} Jun 03 16:44:09.322640 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.322619 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:44:09.323047 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.323027 2578 generic.go:358] "Generic (PLEG): container finished" podID="b54328d7-952f-4cf4-a355-b6a2a697354c" containerID="792f9fc88c4de7e811221708588c047e21b8e38fbd3e5eb3dfeb6b75d472865a" exitCode=1 Jun 03 16:44:09.323144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.323054 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"f6226a63368396f1aa2ff67fbeb0d5df1f6fe6d92eded3ccf26d5c8439436766"} Jun 03 16:44:09.323144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.323080 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"326646471ea6c66833b316692919ef83ffabb17c35a94def01313cfb876fba1f"} Jun 03 16:44:09.323144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.323095 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerDied","Data":"792f9fc88c4de7e811221708588c047e21b8e38fbd3e5eb3dfeb6b75d472865a"} Jun 03 16:44:09.323144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.323108 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"1136a10320bc4f769cb417869b0ac503268f1ddfe282dc1a7d1a47d4a66b1bcf"} Jun 03 16:44:09.324532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.324491 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-251.ec2.internal" podStartSLOduration=20.324478689 podStartE2EDuration="20.324478689s" podCreationTimestamp="2026-06-03 16:43:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:43:52.286015672 +0000 UTC m=+4.669363092" watchObservedRunningTime="2026-06-03 16:44:09.324478689 +0000 UTC m=+21.707826108" Jun 03 16:44:09.324612 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.324588 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4nbf5" podStartSLOduration=3.230331878 podStartE2EDuration="21.324582729s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.60496503 +0000 UTC m=+2.988312433" lastFinishedPulling="2026-06-03 16:44:08.699215881 +0000 UTC m=+21.082563284" observedRunningTime="2026-06-03 16:44:09.324174279 +0000 UTC m=+21.707521700" watchObservedRunningTime="2026-06-03 16:44:09.324582729 +0000 UTC m=+21.707930147" Jun 03 16:44:09.339278 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.339240 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ht9gl" podStartSLOduration=11.313991748 podStartE2EDuration="21.33922498s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.597722349 +0000 UTC m=+2.981069760" lastFinishedPulling="2026-06-03 16:44:00.622955583 +0000 UTC m=+13.006302992" observedRunningTime="2026-06-03 16:44:09.33847224 +0000 UTC m=+21.721819672" watchObservedRunningTime="2026-06-03 16:44:09.33922498 +0000 UTC m=+21.722572403" Jun 03 16:44:09.351748 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.351707 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-xmbrr" podStartSLOduration=3.273148188 podStartE2EDuration="21.351692374s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.59921257 +0000 UTC m=+2.982559974" lastFinishedPulling="2026-06-03 16:44:08.67775676 +0000 UTC m=+21.061104160" observedRunningTime="2026-06-03 16:44:09.350839393 +0000 UTC m=+21.734186812" watchObservedRunningTime="2026-06-03 16:44:09.351692374 +0000 UTC m=+21.735039797" Jun 03 16:44:09.390199 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.390063 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ql6tk" podStartSLOduration=3.283732692 podStartE2EDuration="21.390046944s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.593169952 +0000 UTC m=+2.976517367" lastFinishedPulling="2026-06-03 16:44:08.699484206 +0000 UTC m=+21.082831619" observedRunningTime="2026-06-03 16:44:09.38955155 +0000 UTC m=+21.772898972" watchObservedRunningTime="2026-06-03 16:44:09.390046944 +0000 UTC m=+21.773394365" Jun 03 16:44:09.390199 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.390184 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l74wk" podStartSLOduration=3.312450673 podStartE2EDuration="21.390178825s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.600356616 +0000 UTC m=+2.983704014" lastFinishedPulling="2026-06-03 16:44:08.678084755 +0000 UTC m=+21.061432166" observedRunningTime="2026-06-03 16:44:09.369192324 +0000 UTC m=+21.752539744" watchObservedRunningTime="2026-06-03 16:44:09.390178825 +0000 UTC m=+21.773526245" Jun 03 16:44:09.972842 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:09.972820 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 16:44:10.047089 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.047016 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T16:44:09.972838738Z","UUID":"2c6c89ae-8caa-4ded-9b5b-239dec39aab9","Handler":null,"Name":"","Endpoint":""} Jun 03 16:44:10.049406 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.049389 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 16:44:10.049538 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.049412 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 16:44:10.095499 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.095438 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:10.095643 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:10.095534 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:10.095643 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.095588 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:10.095761 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:10.095702 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:10.326385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.326350 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" event={"ID":"ed72f769-6bc9-433b-9fb4-58960af9908a","Type":"ContainerStarted","Data":"f594e4ca1c4f48cdf5dddc8b874ba9a6627fb30316b08decc320d0d41c8d41bd"} Jun 03 16:44:10.327639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.327617 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zsc6l" event={"ID":"d4364c3c-6642-4620-9776-3dc36f3526aa","Type":"ContainerStarted","Data":"ac69c98a804fd7147754ff9b18ed374fcb46cd5ac864e2e508a1e8a415cb531d"} Jun 03 16:44:10.330011 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.329993 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:44:10.330279 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.330258 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"31e374a63964085ad491692363512a335cdb39fe56b532fb53d15648d48c3db5"} Jun 03 16:44:10.330387 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.330284 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"c61c57dd2e5d76fc60f6846fb959c3416448057e9e2de91a8b25d660fdfa3a9c"} Jun 03 16:44:10.341052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:10.341017 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zsc6l" podStartSLOduration=4.254912446 podStartE2EDuration="22.341007462s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.591662541 +0000 UTC m=+2.975009945" lastFinishedPulling="2026-06-03 16:44:08.67775756 +0000 UTC m=+21.061104961" observedRunningTime="2026-06-03 16:44:10.340867538 +0000 UTC m=+22.724214958" watchObservedRunningTime="2026-06-03 16:44:10.341007462 +0000 UTC m=+22.724354881" Jun 03 16:44:11.334571 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:11.334483 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" event={"ID":"ed72f769-6bc9-433b-9fb4-58960af9908a","Type":"ContainerStarted","Data":"a8deb297c207847d4346364e455148f9fccb70f5c2d74022173b5524e61bf1a1"} Jun 03 16:44:11.353705 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:11.353659 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x8c5n" podStartSLOduration=3.07224382 podStartE2EDuration="23.3536462s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.596437819 +0000 UTC m=+2.979785228" lastFinishedPulling="2026-06-03 16:44:10.877840197 +0000 UTC m=+23.261187608" observedRunningTime="2026-06-03 16:44:11.35341068 +0000 UTC m=+23.736758098" watchObservedRunningTime="2026-06-03 16:44:11.3536462 +0000 UTC m=+23.736993619" Jun 03 16:44:12.095490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:12.095448 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:12.095684 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:12.095607 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:12.095684 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:12.095646 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:12.095808 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:12.095714 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:12.338549 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:12.338512 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:44:12.338964 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:12.338877 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"0a439efe385b0456239afd53f1d1f3eb9709543889c11e98df266109fc60c41a"} Jun 03 16:44:13.465611 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:13.465580 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:44:13.675553 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:13.675530 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:44:13.676333 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:13.676316 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:44:14.094819 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:14.094733 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:14.094969 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:14.094736 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:14.094969 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:14.094850 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:14.095091 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:14.094964 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:14.345628 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:14.345542 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:44:14.346057 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:14.346023 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"6f0b8a4273fb60d79b647083f7fb8ee2281c3df38d0a95ccc5a98989c0a7e4d4"} Jun 03 16:44:14.346641 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:14.346616 2578 scope.go:117] "RemoveContainer" containerID="792f9fc88c4de7e811221708588c047e21b8e38fbd3e5eb3dfeb6b75d472865a" Jun 03 16:44:14.346835 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:14.346797 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ht9gl" Jun 03 16:44:15.351868 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.351639 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:44:15.352544 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.352271 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" event={"ID":"b54328d7-952f-4cf4-a355-b6a2a697354c","Type":"ContainerStarted","Data":"20b1161024f38ee7a2296b23f0c7b5e8617ab24d7195c03a26cbe604d9f2b897"} Jun 03 16:44:15.352544 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.352461 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 03 16:44:15.352931 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.352911 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:44:15.353038 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.352940 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:44:15.370548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.370522 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:44:15.371124 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.371094 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:44:15.546708 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.546619 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" podStartSLOduration=9.387087775 podStartE2EDuration="27.546601645s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.605649513 +0000 UTC m=+2.988996909" lastFinishedPulling="2026-06-03 16:44:08.765163374 +0000 UTC m=+21.148510779" observedRunningTime="2026-06-03 16:44:15.488782411 +0000 UTC m=+27.872129830" watchObservedRunningTime="2026-06-03 16:44:15.546601645 +0000 UTC m=+27.929949063" Jun 03 16:44:15.614404 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.614377 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9sggr"] Jun 03 16:44:15.614511 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.614487 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:15.614596 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:15.614578 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:15.616462 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.616439 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rn64j"] Jun 03 16:44:15.616551 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:15.616536 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:15.616645 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:15.616628 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:16.355689 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:16.355658 2578 generic.go:358] "Generic (PLEG): container finished" podID="4f5d7b02-bc02-4e14-8aa3-5b489f48826c" containerID="e1f7c7dc4c1486f8e39c982eed367aa817c98b172ca226fc4a7809a54613b6b6" exitCode=0 Jun 03 16:44:16.356071 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:16.355761 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerDied","Data":"e1f7c7dc4c1486f8e39c982eed367aa817c98b172ca226fc4a7809a54613b6b6"} Jun 03 16:44:16.356071 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:16.356050 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 03 16:44:16.789903 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:16.789822 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:44:17.094532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:17.094471 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:17.094532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:17.094502 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:17.094705 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:17.094603 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:17.094768 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:17.094722 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:17.358961 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:17.358893 2578 generic.go:358] "Generic (PLEG): container finished" podID="4f5d7b02-bc02-4e14-8aa3-5b489f48826c" containerID="6bab171c0ebb8bd7560c40196413726dfbf521df184f9f6ba12583246f721bf8" exitCode=0 Jun 03 16:44:17.359296 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:17.358973 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerDied","Data":"6bab171c0ebb8bd7560c40196413726dfbf521df184f9f6ba12583246f721bf8"} Jun 03 16:44:18.374119 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:18.373900 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" podUID="b54328d7-952f-4cf4-a355-b6a2a697354c" containerName="ovnkube-controller" probeResult="failure" output="" Jun 03 16:44:19.094721 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.094690 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:19.094878 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.094696 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:19.094878 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.094793 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:19.094878 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.094865 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:19.121079 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.119674 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dkxz2"] Jun 03 16:44:19.123915 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.123897 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.124009 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.123948 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkxz2" podUID="dcbc39e8-0839-44bd-9fa3-c757d3900a12" Jun 03 16:44:19.127697 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.127677 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkxz2"] Jun 03 16:44:19.222345 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.222314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcbc39e8-0839-44bd-9fa3-c757d3900a12-dbus\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.222475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.222357 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcbc39e8-0839-44bd-9fa3-c757d3900a12-kubelet-config\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.222475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.222402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.322883 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.322855 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcbc39e8-0839-44bd-9fa3-c757d3900a12-dbus\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.323003 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.322886 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcbc39e8-0839-44bd-9fa3-c757d3900a12-kubelet-config\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.323003 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.322906 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.323003 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.322991 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcbc39e8-0839-44bd-9fa3-c757d3900a12-kubelet-config\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.323093 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.323012 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:19.323093 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.323063 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret podName:dcbc39e8-0839-44bd-9fa3-c757d3900a12 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:19.823050189 +0000 UTC m=+32.206397585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret") pod "global-pull-secret-syncer-dkxz2" (UID: "dcbc39e8-0839-44bd-9fa3-c757d3900a12") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:19.323162 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.323112 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcbc39e8-0839-44bd-9fa3-c757d3900a12-dbus\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.364919 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.364854 2578 generic.go:358] "Generic (PLEG): container finished" podID="4f5d7b02-bc02-4e14-8aa3-5b489f48826c" containerID="8931aec4e538c77faa0ae7abe3b5a551e0b1999f9ccfce7e311fe1721048f056" exitCode=0 Jun 03 16:44:19.365011 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.364923 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.365011 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.364935 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerDied","Data":"8931aec4e538c77faa0ae7abe3b5a551e0b1999f9ccfce7e311fe1721048f056"} Jun 03 16:44:19.365011 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.364997 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkxz2" podUID="dcbc39e8-0839-44bd-9fa3-c757d3900a12" Jun 03 16:44:19.827091 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:19.827021 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:19.827490 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.827141 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:19.827490 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:19.827192 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret podName:dcbc39e8-0839-44bd-9fa3-c757d3900a12 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:20.827177275 +0000 UTC m=+33.210524671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret") pod "global-pull-secret-syncer-dkxz2" (UID: "dcbc39e8-0839-44bd-9fa3-c757d3900a12") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:20.833747 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:20.833720 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:20.834087 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:20.833852 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:20.834087 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:20.833912 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret podName:dcbc39e8-0839-44bd-9fa3-c757d3900a12 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:22.833895594 +0000 UTC m=+35.217242995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret") pod "global-pull-secret-syncer-dkxz2" (UID: "dcbc39e8-0839-44bd-9fa3-c757d3900a12") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:21.095397 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:21.095319 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:21.095397 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:21.095333 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:21.095397 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:21.095333 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:21.095786 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.095422 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkxz2" podUID="dcbc39e8-0839-44bd-9fa3-c757d3900a12" Jun 03 16:44:21.095786 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.095512 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sggr" podUID="d37d2808-e198-47f9-b0fb-06f4f9ee318d" Jun 03 16:44:21.095786 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.095615 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:44:21.740445 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:21.740410 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:21.740611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.740570 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:44:21.740611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.740590 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:44:21.740611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.740603 2578 projected.go:194] Error preparing data for projected volume kube-api-access-vfzdr for pod openshift-network-diagnostics/network-check-target-9sggr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:44:21.740748 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.740663 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr podName:d37d2808-e198-47f9-b0fb-06f4f9ee318d nodeName:}" failed. No retries permitted until 2026-06-03 16:44:53.740642893 +0000 UTC m=+66.123990309 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfzdr" (UniqueName: "kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr") pod "network-check-target-9sggr" (UID: "d37d2808-e198-47f9-b0fb-06f4f9ee318d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:44:21.841089 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:21.841061 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:21.841512 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.841174 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:44:21.841512 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:21.841233 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:44:53.841217944 +0000 UTC m=+66.224565343 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:44:22.469847 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.469825 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-251.ec2.internal" event="NodeReady" Jun 03 16:44:22.469993 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.469964 2578 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 16:44:22.516347 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.516314 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x82rl"] Jun 03 16:44:22.519419 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.519401 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tch6r"] Jun 03 16:44:22.519596 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.519579 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.521337 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.521316 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 16:44:22.521337 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.521324 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 16:44:22.521465 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.521319 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6lvq\"" Jun 03 16:44:22.522144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.522130 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:22.523752 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.523739 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 16:44:22.523858 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.523842 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 16:44:22.523900 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.523884 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 16:44:22.523968 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.523951 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vw2lw\"" Jun 03 16:44:22.528330 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.528313 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x82rl"] Jun 03 16:44:22.531155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.531136 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tch6r"] Jun 03 16:44:22.647261 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.647192 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba615b97-7ad1-40fe-868e-e6ec99612afe-tmp-dir\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.647379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.647261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba615b97-7ad1-40fe-868e-e6ec99612afe-config-volume\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.647379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.647287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6z2p\" (UniqueName: \"kubernetes.io/projected/a904f235-adff-4af5-b66c-fee8a13025d5-kube-api-access-x6z2p\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:22.647379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.647310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.647379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.647355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qbdp\" (UniqueName: \"kubernetes.io/projected/ba615b97-7ad1-40fe-868e-e6ec99612afe-kube-api-access-6qbdp\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.647379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.647374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:22.748584 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.748543 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.748711 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.748629 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6qbdp\" (UniqueName: \"kubernetes.io/projected/ba615b97-7ad1-40fe-868e-e6ec99612afe-kube-api-access-6qbdp\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.748711 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.748659 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:22.748711 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.748689 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba615b97-7ad1-40fe-868e-e6ec99612afe-tmp-dir\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.748866 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:22.748690 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:22.748866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.748746 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba615b97-7ad1-40fe-868e-e6ec99612afe-config-volume\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.748866 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:22.748742 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:22.748866 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:22.748812 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:44:23.248789014 +0000 UTC m=+35.632136416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:22.748866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.748848 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x6z2p\" (UniqueName: \"kubernetes.io/projected/a904f235-adff-4af5-b66c-fee8a13025d5-kube-api-access-x6z2p\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:22.749117 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:22.748868 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:23.248851064 +0000 UTC m=+35.632198464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:22.749117 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.749011 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba615b97-7ad1-40fe-868e-e6ec99612afe-tmp-dir\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.749358 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.749332 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba615b97-7ad1-40fe-868e-e6ec99612afe-config-volume\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.759026 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.759004 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qbdp\" (UniqueName: \"kubernetes.io/projected/ba615b97-7ad1-40fe-868e-e6ec99612afe-kube-api-access-6qbdp\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:22.759209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.759191 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6z2p\" (UniqueName: \"kubernetes.io/projected/a904f235-adff-4af5-b66c-fee8a13025d5-kube-api-access-x6z2p\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:22.849503 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:22.849458 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:22.850153 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:22.849644 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:22.850153 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:22.849728 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret podName:dcbc39e8-0839-44bd-9fa3-c757d3900a12 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:26.849710342 +0000 UTC m=+39.233057739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret") pod "global-pull-secret-syncer-dkxz2" (UID: "dcbc39e8-0839-44bd-9fa3-c757d3900a12") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:44:23.095373 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.095301 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:23.095504 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.095475 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:23.095658 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.095628 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:23.097883 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.097861 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:44:23.097989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.097937 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h6q5l\"" Jun 03 16:44:23.097989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.097975 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:44:23.098104 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.097979 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:44:23.098104 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.097979 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6fvvv\"" Jun 03 16:44:23.098349 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.098336 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 16:44:23.252811 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.252779 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:23.252979 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:23.252882 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:23.252979 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:23.252953 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:24.252937635 +0000 UTC m=+36.636285035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:23.253094 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:23.252977 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:23.253094 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:23.253044 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:23.253094 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:23.253075 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:44:24.253067261 +0000 UTC m=+36.636414657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:24.257702 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:24.257669 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:24.258271 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:24.257758 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:24.258271 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:24.257840 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:24.258271 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:24.257874 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:24.258271 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:24.257907 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:26.257888429 +0000 UTC m=+38.641235826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:24.258271 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:24.257933 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:44:26.257917972 +0000 UTC m=+38.641265373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:26.273203 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:26.273156 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:26.273700 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:26.273337 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:26.273700 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:26.273405 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:26.273700 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:26.273422 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:30.273400295 +0000 UTC m=+42.656747695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:26.273700 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:26.273500 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:26.273700 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:26.273571 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:44:30.273540782 +0000 UTC m=+42.656888179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:26.877829 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:26.877792 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:26.879987 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:26.879955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcbc39e8-0839-44bd-9fa3-c757d3900a12-original-pull-secret\") pod \"global-pull-secret-syncer-dkxz2\" (UID: \"dcbc39e8-0839-44bd-9fa3-c757d3900a12\") " pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:27.005660 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:27.005631 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkxz2" Jun 03 16:44:27.320688 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:27.320665 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkxz2"] Jun 03 16:44:27.323477 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:44:27.323450 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbc39e8_0839_44bd_9fa3_c757d3900a12.slice/crio-80e9812cfdb276e1458cd4d1a20310a0b6a308ce62aebf66c0a589b442162cc2 WatchSource:0}: Error finding container 80e9812cfdb276e1458cd4d1a20310a0b6a308ce62aebf66c0a589b442162cc2: Status 404 returned error can't find the container with id 80e9812cfdb276e1458cd4d1a20310a0b6a308ce62aebf66c0a589b442162cc2 Jun 03 16:44:27.383452 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:27.383418 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerStarted","Data":"ee7c32fdb1f5ebfc10b652dc402241ab8d854c7402a7ab5e6339301bc630d1d6"} Jun 03 16:44:27.384381 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:27.384358 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkxz2" event={"ID":"dcbc39e8-0839-44bd-9fa3-c757d3900a12","Type":"ContainerStarted","Data":"80e9812cfdb276e1458cd4d1a20310a0b6a308ce62aebf66c0a589b442162cc2"} Jun 03 16:44:28.389648 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:28.389611 2578 generic.go:358] "Generic (PLEG): container finished" podID="4f5d7b02-bc02-4e14-8aa3-5b489f48826c" containerID="ee7c32fdb1f5ebfc10b652dc402241ab8d854c7402a7ab5e6339301bc630d1d6" exitCode=0 Jun 03 16:44:28.390035 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:28.389679 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerDied","Data":"ee7c32fdb1f5ebfc10b652dc402241ab8d854c7402a7ab5e6339301bc630d1d6"} Jun 03 16:44:29.394497 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:29.394460 2578 generic.go:358] "Generic (PLEG): container finished" podID="4f5d7b02-bc02-4e14-8aa3-5b489f48826c" containerID="bf19fd7be1d9f0647d0675fca3725963c394d292be3f7feca61a1e833a956f7d" exitCode=0 Jun 03 16:44:29.394892 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:29.394537 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerDied","Data":"bf19fd7be1d9f0647d0675fca3725963c394d292be3f7feca61a1e833a956f7d"} Jun 03 16:44:30.302364 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:30.302326 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:30.302550 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:30.302394 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:30.302550 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:30.302501 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:30.302550 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:30.302524 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:30.302719 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:30.302602 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:44:38.302580171 +0000 UTC m=+50.685927568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:30.302719 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:30.302628 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:38.302616164 +0000 UTC m=+50.685963560 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:30.400263 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:30.400230 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" event={"ID":"4f5d7b02-bc02-4e14-8aa3-5b489f48826c","Type":"ContainerStarted","Data":"222f363f888da28fbc2e26fa7f392395bdda93e947de6b4e4a0ff3d32c8a28e0"} Jun 03 16:44:30.422320 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:30.422262 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8qqp5" podStartSLOduration=5.827840714 podStartE2EDuration="42.422244625s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:43:50.593976615 +0000 UTC m=+2.977324030" lastFinishedPulling="2026-06-03 16:44:27.188380539 +0000 UTC m=+39.571727941" observedRunningTime="2026-06-03 16:44:30.421195764 +0000 UTC m=+42.804543184" watchObservedRunningTime="2026-06-03 16:44:30.422244625 +0000 UTC m=+42.805592045" Jun 03 16:44:33.406928 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:33.406894 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkxz2" event={"ID":"dcbc39e8-0839-44bd-9fa3-c757d3900a12","Type":"ContainerStarted","Data":"d128b0253411a1b5eee3ee8deb829ddbc130801ccddda8e9fd38eccf88cb113b"} Jun 03 16:44:33.422193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:33.422151 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dkxz2" podStartSLOduration=9.155526433 podStartE2EDuration="14.422138682s" podCreationTimestamp="2026-06-03 16:44:19 +0000 UTC" firstStartedPulling="2026-06-03 16:44:27.325306579 +0000 UTC m=+39.708653976" lastFinishedPulling="2026-06-03 16:44:32.591918827 +0000 UTC m=+44.975266225" observedRunningTime="2026-06-03 16:44:33.421696858 +0000 UTC m=+45.805044277" watchObservedRunningTime="2026-06-03 16:44:33.422138682 +0000 UTC m=+45.805486100" Jun 03 16:44:38.356510 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:38.356480 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:38.357014 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:38.356528 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:38.357014 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:38.356640 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:38.357014 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:38.356646 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:38.357014 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:38.356691 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:44:54.356677477 +0000 UTC m=+66.740024879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:38.357014 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:38.356703 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:44:54.356697782 +0000 UTC m=+66.740045178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:48.370411 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:48.370382 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lp8cj" Jun 03 16:44:53.750538 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:53.750494 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:53.752700 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:53.752680 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:44:53.763031 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:53.763013 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:44:53.773974 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:53.773953 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfzdr\" (UniqueName: \"kubernetes.io/projected/d37d2808-e198-47f9-b0fb-06f4f9ee318d-kube-api-access-vfzdr\") pod \"network-check-target-9sggr\" (UID: \"d37d2808-e198-47f9-b0fb-06f4f9ee318d\") " pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:53.850797 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:53.850767 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:44:53.853061 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:53.853046 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:44:53.861588 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:53.861572 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 16:44:53.861634 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:53.861628 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:45:57.861612515 +0000 UTC m=+130.244959912 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : secret "metrics-daemon-secret" not found Jun 03 16:44:54.012481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:54.012427 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h6q5l\"" Jun 03 16:44:54.020085 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:54.020070 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:54.133936 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:54.133911 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9sggr"] Jun 03 16:44:54.137234 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:44:54.137206 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37d2808_e198_47f9_b0fb_06f4f9ee318d.slice/crio-19eb1a8921424101a67221979e0192911accd319d51d8796d4de96f156bc6846 WatchSource:0}: Error finding container 19eb1a8921424101a67221979e0192911accd319d51d8796d4de96f156bc6846: Status 404 returned error can't find the container with id 19eb1a8921424101a67221979e0192911accd319d51d8796d4de96f156bc6846 Jun 03 16:44:54.445862 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:54.445832 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9sggr" event={"ID":"d37d2808-e198-47f9-b0fb-06f4f9ee318d","Type":"ContainerStarted","Data":"19eb1a8921424101a67221979e0192911accd319d51d8796d4de96f156bc6846"} Jun 03 16:44:54.454078 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:54.454057 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:44:54.454139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:54.454103 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:44:54.454197 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:54.454184 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:44:54.454231 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:54.454209 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:44:54.454263 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:54.454243 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:45:26.454229757 +0000 UTC m=+98.837577154 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:44:54.454263 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:44:54.454257 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:45:26.454250998 +0000 UTC m=+98.837598395 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:44:57.453781 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:57.453742 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9sggr" event={"ID":"d37d2808-e198-47f9-b0fb-06f4f9ee318d","Type":"ContainerStarted","Data":"808bf8ee7252c724177a3416dadecb808fb40ad5f19af50875ff21229eb9d7be"} Jun 03 16:44:57.454162 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:57.453884 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:44:57.468019 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:44:57.467950 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9sggr" podStartSLOduration=66.915252855 podStartE2EDuration="1m9.467935711s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:44:54.138905299 +0000 UTC m=+66.522252695" lastFinishedPulling="2026-06-03 16:44:56.69158814 +0000 UTC m=+69.074935551" observedRunningTime="2026-06-03 16:44:57.467533611 +0000 UTC m=+69.850881032" watchObservedRunningTime="2026-06-03 16:44:57.467935711 +0000 UTC m=+69.851283132" Jun 03 16:45:26.458574 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:45:26.458517 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:45:26.459080 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:45:26.458619 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:45:26.459080 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:45:26.458664 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:45:26.459080 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:45:26.458741 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:46:30.458724541 +0000 UTC m=+162.842071938 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:45:26.459080 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:45:26.458761 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:45:26.459080 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:45:26.458821 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:30.458805432 +0000 UTC m=+162.842152830 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:45:28.458694 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:45:28.458666 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9sggr" Jun 03 16:45:57.955023 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:45:57.954987 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:45:57.955482 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:45:57.955102 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 16:45:57.955482 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:45:57.955166 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs podName:e0cf9100-e9b8-42a0-93a8-f273610a87cf nodeName:}" failed. No retries permitted until 2026-06-03 16:47:59.955151157 +0000 UTC m=+252.338498558 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs") pod "network-metrics-daemon-rn64j" (UID: "e0cf9100-e9b8-42a0-93a8-f273610a87cf") : secret "metrics-daemon-secret" not found Jun 03 16:46:20.344670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.344632 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht"] Jun 03 16:46:20.347607 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.347586 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:20.349326 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.349302 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kpxlq\"" Jun 03 16:46:20.349416 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.349341 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:46:20.349640 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.349622 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 03 16:46:20.349640 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.349631 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 03 16:46:20.356523 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.356498 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht"] Jun 03 16:46:20.448964 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.448941 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7"] Jun 03 16:46:20.451773 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.451755 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8488d7768f-pwvnk"] Jun 03 16:46:20.451904 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.451888 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.454265 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.454249 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.454458 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.454437 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 03 16:46:20.454751 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.454733 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-s9f5b\"" Jun 03 16:46:20.454851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.454787 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 03 16:46:20.454851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.454738 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:46:20.454965 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.454850 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 03 16:46:20.456376 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.456358 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 16:46:20.456766 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.456750 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 16:46:20.458835 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.458807 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9p26t\"" Jun 03 16:46:20.460276 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.460262 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 16:46:20.462173 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.462154 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7"] Jun 03 16:46:20.466026 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.466007 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 16:46:20.468497 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.468479 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8488d7768f-pwvnk"] Jun 03 16:46:20.502254 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.502227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:20.502353 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.502263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjn9n\" (UniqueName: \"kubernetes.io/projected/945eb8a0-9d7a-4e74-b114-a253cf36dda9-kube-api-access-xjn9n\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:20.542193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.542166 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn"] Jun 03 16:46:20.545071 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.545055 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.546796 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.546774 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr"] Jun 03 16:46:20.547754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.547738 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 03 16:46:20.547998 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.547986 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9w2ql\"" Jun 03 16:46:20.548749 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.548727 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 16:46:20.548964 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.548939 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 16:46:20.549285 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.549268 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-788bf58df5-5n57j"] Jun 03 16:46:20.549405 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.549391 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" Jun 03 16:46:20.550437 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.550418 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 03 16:46:20.551945 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.551929 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.552402 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.552388 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jv784\"" Jun 03 16:46:20.553546 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.553518 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 03 16:46:20.553546 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.553568 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 16:46:20.553747 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.553619 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 16:46:20.553832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.553779 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xwljj\"" Jun 03 16:46:20.554041 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.554026 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 03 16:46:20.555774 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.555756 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn"] Jun 03 16:46:20.560466 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.560439 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr"] Jun 03 16:46:20.560691 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.560677 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 03 16:46:20.565582 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.565551 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-5n57j"] Jun 03 16:46:20.602824 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/604f5077-f50a-4f66-8951-8ff2799b3e82-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.602824 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-bound-sa-token\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.602942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-installation-pull-secrets\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.602942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602854 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:20.602942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-certificates\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.603044 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zw5t\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-kube-api-access-8zw5t\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.603044 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.602965 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 03 16:46:20.603044 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.602966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwq4l\" (UniqueName: \"kubernetes.io/projected/604f5077-f50a-4f66-8951-8ff2799b3e82-kube-api-access-qwq4l\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.603044 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.603015 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls podName:945eb8a0-9d7a-4e74-b114-a253cf36dda9 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:21.103001754 +0000 UTC m=+153.486349151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-f9zht" (UID: "945eb8a0-9d7a-4e74-b114-a253cf36dda9") : secret "samples-operator-tls" not found Jun 03 16:46:20.603044 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.603038 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjn9n\" (UniqueName: \"kubernetes.io/projected/945eb8a0-9d7a-4e74-b114-a253cf36dda9-kube-api-access-xjn9n\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:20.603369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.603057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-image-registry-private-configuration\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.603369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.603078 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.603369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.603148 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-trusted-ca\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.603369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.603195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/604f5077-f50a-4f66-8951-8ff2799b3e82-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.603369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.603233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-ca-trust-extracted\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.611828 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.611803 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjn9n\" (UniqueName: \"kubernetes.io/projected/945eb8a0-9d7a-4e74-b114-a253cf36dda9-kube-api-access-xjn9n\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:20.703885 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.703864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgfwb\" (UniqueName: \"kubernetes.io/projected/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-kube-api-access-pgfwb\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.703984 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.703892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5k4m\" (UniqueName: \"kubernetes.io/projected/8acf6cda-67b1-4a7e-964f-154e9bf17441-kube-api-access-m5k4m\") pod \"network-check-source-56d84b9f44-rjzvr\" (UID: \"8acf6cda-67b1-4a7e-964f-154e9bf17441\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" Jun 03 16:46:20.703984 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.703919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71f052c3-b28b-4eff-a502-518f583b7049-snapshots\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.704059 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.703973 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qwq4l\" (UniqueName: \"kubernetes.io/projected/604f5077-f50a-4f66-8951-8ff2799b3e82-kube-api-access-qwq4l\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.704059 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.704059 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71f052c3-b28b-4eff-a502-518f583b7049-tmp\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.704150 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704060 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/604f5077-f50a-4f66-8951-8ff2799b3e82-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.704150 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zdsx\" (UniqueName: \"kubernetes.io/projected/71f052c3-b28b-4eff-a502-518f583b7049-kube-api-access-9zdsx\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.704150 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704103 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-ca-trust-extracted\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704238 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704183 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-bound-sa-token\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704238 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704203 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-installation-pull-secrets\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704238 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704220 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-certificates\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704380 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704250 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8zw5t\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-kube-api-access-8zw5t\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704380 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71f052c3-b28b-4eff-a502-518f583b7049-serving-cert\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.704481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704393 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-image-registry-private-configuration\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704422 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704463 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-trusted-ca\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704465 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-ca-trust-extracted\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f052c3-b28b-4eff-a502-518f583b7049-service-ca-bundle\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.704581 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.704598 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8488d7768f-pwvnk: secret "image-registry-tls" not found Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.704647 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls podName:1bc9511e-e38e-40eb-bd0e-8b038c20a4b7 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:21.20463003 +0000 UTC m=+153.587977456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls") pod "image-registry-8488d7768f-pwvnk" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7") : secret "image-registry-tls" not found Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704706 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/604f5077-f50a-4f66-8951-8ff2799b3e82-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.704740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f052c3-b28b-4eff-a502-518f583b7049-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.705052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704783 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/604f5077-f50a-4f66-8951-8ff2799b3e82-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.705052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.704924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-certificates\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.705244 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.705227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-trusted-ca\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.706715 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.706691 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-installation-pull-secrets\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.706821 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.706736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/604f5077-f50a-4f66-8951-8ff2799b3e82-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.706821 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.706755 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-image-registry-private-configuration\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.712343 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.712324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwq4l\" (UniqueName: \"kubernetes.io/projected/604f5077-f50a-4f66-8951-8ff2799b3e82-kube-api-access-qwq4l\") pod \"kube-storage-version-migrator-operator-96ddc787c-th5r7\" (UID: \"604f5077-f50a-4f66-8951-8ff2799b3e82\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.712458 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.712405 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-bound-sa-token\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.712650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.712633 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zw5t\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-kube-api-access-8zw5t\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:20.761708 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.761686 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" Jun 03 16:46:20.805716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805687 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f052c3-b28b-4eff-a502-518f583b7049-service-ca-bundle\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.805851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805728 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.805851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805751 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f052c3-b28b-4eff-a502-518f583b7049-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.805851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805783 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pgfwb\" (UniqueName: \"kubernetes.io/projected/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-kube-api-access-pgfwb\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.805851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805806 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k4m\" (UniqueName: \"kubernetes.io/projected/8acf6cda-67b1-4a7e-964f-154e9bf17441-kube-api-access-m5k4m\") pod \"network-check-source-56d84b9f44-rjzvr\" (UID: \"8acf6cda-67b1-4a7e-964f-154e9bf17441\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" Jun 03 16:46:20.805851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805842 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71f052c3-b28b-4eff-a502-518f583b7049-snapshots\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.806117 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805875 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.806117 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805911 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71f052c3-b28b-4eff-a502-518f583b7049-tmp\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.806117 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805932 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9zdsx\" (UniqueName: \"kubernetes.io/projected/71f052c3-b28b-4eff-a502-518f583b7049-kube-api-access-9zdsx\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.806117 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.805963 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71f052c3-b28b-4eff-a502-518f583b7049-serving-cert\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.806383 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.806360 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f052c3-b28b-4eff-a502-518f583b7049-service-ca-bundle\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.807130 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.806721 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:20.807130 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:20.806787 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls podName:d8bd50dd-7ddd-4a74-ae63-c6114a93388e nodeName:}" failed. No retries permitted until 2026-06-03 16:46:21.306767499 +0000 UTC m=+153.690114902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-2lgjn" (UID: "d8bd50dd-7ddd-4a74-ae63-c6114a93388e") : secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:20.807130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.807095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.807130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.807117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71f052c3-b28b-4eff-a502-518f583b7049-snapshots\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.807398 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.807341 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71f052c3-b28b-4eff-a502-518f583b7049-tmp\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.807494 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.807467 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f052c3-b28b-4eff-a502-518f583b7049-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.808860 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.808835 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71f052c3-b28b-4eff-a502-518f583b7049-serving-cert\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.817474 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.817444 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5k4m\" (UniqueName: \"kubernetes.io/projected/8acf6cda-67b1-4a7e-964f-154e9bf17441-kube-api-access-m5k4m\") pod \"network-check-source-56d84b9f44-rjzvr\" (UID: \"8acf6cda-67b1-4a7e-964f-154e9bf17441\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" Jun 03 16:46:20.818165 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.818011 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgfwb\" (UniqueName: \"kubernetes.io/projected/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-kube-api-access-pgfwb\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:20.818380 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.818304 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zdsx\" (UniqueName: \"kubernetes.io/projected/71f052c3-b28b-4eff-a502-518f583b7049-kube-api-access-9zdsx\") pod \"insights-operator-788bf58df5-5n57j\" (UID: \"71f052c3-b28b-4eff-a502-518f583b7049\") " pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.862955 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.862905 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" Jun 03 16:46:20.868613 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.868591 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-5n57j" Jun 03 16:46:20.876104 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.876079 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7"] Jun 03 16:46:20.879062 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:20.879036 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod604f5077_f50a_4f66_8951_8ff2799b3e82.slice/crio-6c8369fae431f6d1a5edb7b1eed2a599fa32883346538059384a2c7a1cb8d8f2 WatchSource:0}: Error finding container 6c8369fae431f6d1a5edb7b1eed2a599fa32883346538059384a2c7a1cb8d8f2: Status 404 returned error can't find the container with id 6c8369fae431f6d1a5edb7b1eed2a599fa32883346538059384a2c7a1cb8d8f2 Jun 03 16:46:20.982422 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:20.982365 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr"] Jun 03 16:46:20.986219 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:20.986191 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acf6cda_67b1_4a7e_964f_154e9bf17441.slice/crio-9ec221fca81ec078d311f373a6ee6de2fa2da208cd082df09febcce276a99304 WatchSource:0}: Error finding container 9ec221fca81ec078d311f373a6ee6de2fa2da208cd082df09febcce276a99304: Status 404 returned error can't find the container with id 9ec221fca81ec078d311f373a6ee6de2fa2da208cd082df09febcce276a99304 Jun 03 16:46:21.001324 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.001302 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-5n57j"] Jun 03 16:46:21.005197 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:21.005171 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f052c3_b28b_4eff_a502_518f583b7049.slice/crio-716c9092ed104cf80b393b980b02a4e7dc0527e6c4142ae548db174424d8ebf6 WatchSource:0}: Error finding container 716c9092ed104cf80b393b980b02a4e7dc0527e6c4142ae548db174424d8ebf6: Status 404 returned error can't find the container with id 716c9092ed104cf80b393b980b02a4e7dc0527e6c4142ae548db174424d8ebf6 Jun 03 16:46:21.108247 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.108215 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:21.108392 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.108353 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 03 16:46:21.108429 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.108413 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls podName:945eb8a0-9d7a-4e74-b114-a253cf36dda9 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:22.108394913 +0000 UTC m=+154.491742317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-f9zht" (UID: "945eb8a0-9d7a-4e74-b114-a253cf36dda9") : secret "samples-operator-tls" not found Jun 03 16:46:21.209345 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.209292 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:21.209452 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.209438 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:46:21.209490 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.209460 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8488d7768f-pwvnk: secret "image-registry-tls" not found Jun 03 16:46:21.209523 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.209518 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls podName:1bc9511e-e38e-40eb-bd0e-8b038c20a4b7 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:22.209504176 +0000 UTC m=+154.592851578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls") pod "image-registry-8488d7768f-pwvnk" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7") : secret "image-registry-tls" not found Jun 03 16:46:21.309999 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.309970 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:21.310135 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.310114 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:21.310188 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:21.310182 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls podName:d8bd50dd-7ddd-4a74-ae63-c6114a93388e nodeName:}" failed. No retries permitted until 2026-06-03 16:46:22.310166435 +0000 UTC m=+154.693513844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-2lgjn" (UID: "d8bd50dd-7ddd-4a74-ae63-c6114a93388e") : secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:21.614090 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.614054 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-5n57j" event={"ID":"71f052c3-b28b-4eff-a502-518f583b7049","Type":"ContainerStarted","Data":"716c9092ed104cf80b393b980b02a4e7dc0527e6c4142ae548db174424d8ebf6"} Jun 03 16:46:21.616083 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.616022 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" event={"ID":"8acf6cda-67b1-4a7e-964f-154e9bf17441","Type":"ContainerStarted","Data":"21b998d04fd317ff55b331743432cd630116d487fb3636c9c6fb585735ebeab0"} Jun 03 16:46:21.616083 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.616056 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" event={"ID":"8acf6cda-67b1-4a7e-964f-154e9bf17441","Type":"ContainerStarted","Data":"9ec221fca81ec078d311f373a6ee6de2fa2da208cd082df09febcce276a99304"} Jun 03 16:46:21.617393 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.617368 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" event={"ID":"604f5077-f50a-4f66-8951-8ff2799b3e82","Type":"ContainerStarted","Data":"6c8369fae431f6d1a5edb7b1eed2a599fa32883346538059384a2c7a1cb8d8f2"} Jun 03 16:46:21.632798 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:21.632006 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-rjzvr" podStartSLOduration=1.6319903980000001 podStartE2EDuration="1.631990398s" podCreationTimestamp="2026-06-03 16:46:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:46:21.631293988 +0000 UTC m=+154.014641408" watchObservedRunningTime="2026-06-03 16:46:21.631990398 +0000 UTC m=+154.015337819" Jun 03 16:46:22.116005 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:22.115969 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:22.116175 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.116119 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 03 16:46:22.116251 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.116191 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls podName:945eb8a0-9d7a-4e74-b114-a253cf36dda9 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:24.116169311 +0000 UTC m=+156.499516708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-f9zht" (UID: "945eb8a0-9d7a-4e74-b114-a253cf36dda9") : secret "samples-operator-tls" not found Jun 03 16:46:22.216331 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:22.216302 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:22.216497 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.216480 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:46:22.216571 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.216503 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8488d7768f-pwvnk: secret "image-registry-tls" not found Jun 03 16:46:22.216623 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.216580 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls podName:1bc9511e-e38e-40eb-bd0e-8b038c20a4b7 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:24.216547731 +0000 UTC m=+156.599895132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls") pod "image-registry-8488d7768f-pwvnk" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7") : secret "image-registry-tls" not found Jun 03 16:46:22.317152 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:22.317124 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:22.317286 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.317272 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:22.317344 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:22.317334 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls podName:d8bd50dd-7ddd-4a74-ae63-c6114a93388e nodeName:}" failed. No retries permitted until 2026-06-03 16:46:24.317317315 +0000 UTC m=+156.700664722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-2lgjn" (UID: "d8bd50dd-7ddd-4a74-ae63-c6114a93388e") : secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:24.133669 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.133638 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:24.133989 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.133742 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 03 16:46:24.133989 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.133789 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls podName:945eb8a0-9d7a-4e74-b114-a253cf36dda9 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:28.133775873 +0000 UTC m=+160.517123270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-f9zht" (UID: "945eb8a0-9d7a-4e74-b114-a253cf36dda9") : secret "samples-operator-tls" not found Jun 03 16:46:24.234047 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.234016 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:24.234186 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.234158 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:46:24.234186 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.234174 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8488d7768f-pwvnk: secret "image-registry-tls" not found Jun 03 16:46:24.234256 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.234232 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls podName:1bc9511e-e38e-40eb-bd0e-8b038c20a4b7 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:28.234214545 +0000 UTC m=+160.617561955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls") pod "image-registry-8488d7768f-pwvnk" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7") : secret "image-registry-tls" not found Jun 03 16:46:24.334995 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.334967 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:24.335118 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.335083 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:24.335157 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:24.335129 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls podName:d8bd50dd-7ddd-4a74-ae63-c6114a93388e nodeName:}" failed. No retries permitted until 2026-06-03 16:46:28.33511704 +0000 UTC m=+160.718464442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-2lgjn" (UID: "d8bd50dd-7ddd-4a74-ae63-c6114a93388e") : secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:24.627250 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.627217 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-5n57j" event={"ID":"71f052c3-b28b-4eff-a502-518f583b7049","Type":"ContainerStarted","Data":"350ef60927fae0258cd220f406e62637fd1b3de6e23ddecc3dad038eaa9311a5"} Jun 03 16:46:24.628571 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.628531 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" event={"ID":"604f5077-f50a-4f66-8951-8ff2799b3e82","Type":"ContainerStarted","Data":"a1f5b7a8974f18c0116e21d5376367795fd650a6f941e3dd3058c806358364d6"} Jun 03 16:46:24.643810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.643764 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-788bf58df5-5n57j" podStartSLOduration=1.902258755 podStartE2EDuration="4.643748433s" podCreationTimestamp="2026-06-03 16:46:20 +0000 UTC" firstStartedPulling="2026-06-03 16:46:21.006958004 +0000 UTC m=+153.390305400" lastFinishedPulling="2026-06-03 16:46:23.748447672 +0000 UTC m=+156.131795078" observedRunningTime="2026-06-03 16:46:24.64222904 +0000 UTC m=+157.025576454" watchObservedRunningTime="2026-06-03 16:46:24.643748433 +0000 UTC m=+157.027095848" Jun 03 16:46:24.656658 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:24.656618 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" podStartSLOduration=1.785517294 podStartE2EDuration="4.656606348s" podCreationTimestamp="2026-06-03 16:46:20 +0000 UTC" firstStartedPulling="2026-06-03 16:46:20.881751182 +0000 UTC m=+153.265098582" lastFinishedPulling="2026-06-03 16:46:23.752840236 +0000 UTC m=+156.136187636" observedRunningTime="2026-06-03 16:46:24.655869812 +0000 UTC m=+157.039217232" watchObservedRunningTime="2026-06-03 16:46:24.656606348 +0000 UTC m=+157.039953767" Jun 03 16:46:25.530499 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:25.530415 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-x82rl" podUID="ba615b97-7ad1-40fe-868e-e6ec99612afe" Jun 03 16:46:25.536546 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:25.536520 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-tch6r" podUID="a904f235-adff-4af5-b66c-fee8a13025d5" Jun 03 16:46:25.630658 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:25.630632 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:46:25.630835 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:25.630817 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x82rl" Jun 03 16:46:26.114828 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:26.114789 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rn64j" podUID="e0cf9100-e9b8-42a0-93a8-f273610a87cf" Jun 03 16:46:28.162787 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:28.162757 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:28.163168 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.162874 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 03 16:46:28.163168 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.162923 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls podName:945eb8a0-9d7a-4e74-b114-a253cf36dda9 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:36.162909202 +0000 UTC m=+168.546256599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-f9zht" (UID: "945eb8a0-9d7a-4e74-b114-a253cf36dda9") : secret "samples-operator-tls" not found Jun 03 16:46:28.263258 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:28.263227 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:28.263387 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.263365 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:46:28.263387 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.263383 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8488d7768f-pwvnk: secret "image-registry-tls" not found Jun 03 16:46:28.263459 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.263443 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls podName:1bc9511e-e38e-40eb-bd0e-8b038c20a4b7 nodeName:}" failed. No retries permitted until 2026-06-03 16:46:36.263425133 +0000 UTC m=+168.646772547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls") pod "image-registry-8488d7768f-pwvnk" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7") : secret "image-registry-tls" not found Jun 03 16:46:28.326347 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:28.326326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmbrr_fa1a1e82-367d-49e7-9abb-6f49ef988e5d/dns-node-resolver/0.log" Jun 03 16:46:28.364178 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:28.364154 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:28.364305 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.364290 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:28.364350 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:28.364345 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls podName:d8bd50dd-7ddd-4a74-ae63-c6114a93388e nodeName:}" failed. No retries permitted until 2026-06-03 16:46:36.364328741 +0000 UTC m=+168.747676141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-2lgjn" (UID: "d8bd50dd-7ddd-4a74-ae63-c6114a93388e") : secret "cluster-monitoring-operator-tls" not found Jun 03 16:46:29.327680 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:29.327653 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l74wk_06cc3bd3-c291-4ad8-8515-bb30440a138f/node-ca/0.log" Jun 03 16:46:30.480020 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:30.479973 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:46:30.480020 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:30.480039 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:46:30.480611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:30.480128 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:46:30.480611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:30.480134 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:46:30.480611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:30.480181 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls podName:ba615b97-7ad1-40fe-868e-e6ec99612afe nodeName:}" failed. No retries permitted until 2026-06-03 16:48:32.480167906 +0000 UTC m=+284.863515302 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls") pod "dns-default-x82rl" (UID: "ba615b97-7ad1-40fe-868e-e6ec99612afe") : secret "dns-default-metrics-tls" not found Jun 03 16:46:30.480611 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:30.480195 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert podName:a904f235-adff-4af5-b66c-fee8a13025d5 nodeName:}" failed. No retries permitted until 2026-06-03 16:48:32.480189059 +0000 UTC m=+284.863536455 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert") pod "ingress-canary-tch6r" (UID: "a904f235-adff-4af5-b66c-fee8a13025d5") : secret "canary-serving-cert" not found Jun 03 16:46:30.727464 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:30.727436 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-th5r7_604f5077-f50a-4f66-8951-8ff2799b3e82/kube-storage-version-migrator-operator/0.log" Jun 03 16:46:36.223017 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.222945 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:36.225383 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.225360 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/945eb8a0-9d7a-4e74-b114-a253cf36dda9-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-f9zht\" (UID: \"945eb8a0-9d7a-4e74-b114-a253cf36dda9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:36.257635 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.257610 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" Jun 03 16:46:36.324193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.324086 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:36.326742 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.326724 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"image-registry-8488d7768f-pwvnk\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:36.368676 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.368650 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:36.371421 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.371402 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht"] Jun 03 16:46:36.424519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.424490 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:36.426907 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.426883 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8bd50dd-7ddd-4a74-ae63-c6114a93388e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-2lgjn\" (UID: \"d8bd50dd-7ddd-4a74-ae63-c6114a93388e\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:36.454829 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.454805 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" Jun 03 16:46:36.486355 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.486247 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8488d7768f-pwvnk"] Jun 03 16:46:36.489914 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:36.489883 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc9511e_e38e_40eb_bd0e_8b038c20a4b7.slice/crio-11336b97bcc7cc3392bb9040316db35d19b8b9909e8bbdeb9d2a05afd7de27ac WatchSource:0}: Error finding container 11336b97bcc7cc3392bb9040316db35d19b8b9909e8bbdeb9d2a05afd7de27ac: Status 404 returned error can't find the container with id 11336b97bcc7cc3392bb9040316db35d19b8b9909e8bbdeb9d2a05afd7de27ac Jun 03 16:46:36.576398 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.575641 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn"] Jun 03 16:46:36.578610 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:36.578587 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8bd50dd_7ddd_4a74_ae63_c6114a93388e.slice/crio-191e689cb18d6d4de7b63ea1aaac73e77c0bea3f7f02c0716bce2b027e2b9886 WatchSource:0}: Error finding container 191e689cb18d6d4de7b63ea1aaac73e77c0bea3f7f02c0716bce2b027e2b9886: Status 404 returned error can't find the container with id 191e689cb18d6d4de7b63ea1aaac73e77c0bea3f7f02c0716bce2b027e2b9886 Jun 03 16:46:36.661856 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.661824 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" event={"ID":"d8bd50dd-7ddd-4a74-ae63-c6114a93388e","Type":"ContainerStarted","Data":"191e689cb18d6d4de7b63ea1aaac73e77c0bea3f7f02c0716bce2b027e2b9886"} Jun 03 16:46:36.662811 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.662784 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" event={"ID":"945eb8a0-9d7a-4e74-b114-a253cf36dda9","Type":"ContainerStarted","Data":"c413635d393a3974333a9d190895fff1d5bc8fdda306bb05313cab7261293f74"} Jun 03 16:46:36.663958 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.663939 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" event={"ID":"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7","Type":"ContainerStarted","Data":"3761ee2c3341b5ad56d49144e060e484c5fbba0171771c682e4715a724337eb3"} Jun 03 16:46:36.664059 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.663965 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" event={"ID":"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7","Type":"ContainerStarted","Data":"11336b97bcc7cc3392bb9040316db35d19b8b9909e8bbdeb9d2a05afd7de27ac"} Jun 03 16:46:36.664134 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.664119 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:46:36.688119 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:36.687971 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" podStartSLOduration=16.687957984 podStartE2EDuration="16.687957984s" podCreationTimestamp="2026-06-03 16:46:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:46:36.687194494 +0000 UTC m=+169.070541914" watchObservedRunningTime="2026-06-03 16:46:36.687957984 +0000 UTC m=+169.071305405" Jun 03 16:46:39.095902 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:39.095538 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:46:39.676260 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:39.676229 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" event={"ID":"d8bd50dd-7ddd-4a74-ae63-c6114a93388e","Type":"ContainerStarted","Data":"11004cdfa58b34ffa4b1289098b1a971d458f689824cd254e01883962969f111"} Jun 03 16:46:39.677757 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:39.677732 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" event={"ID":"945eb8a0-9d7a-4e74-b114-a253cf36dda9","Type":"ContainerStarted","Data":"a56bca7a0f9be4d8428a2d8d6a0c30e757a8513b740a8198ae13d7edb9213ebf"} Jun 03 16:46:39.677757 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:39.677759 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" event={"ID":"945eb8a0-9d7a-4e74-b114-a253cf36dda9","Type":"ContainerStarted","Data":"f8a79c335993ae22ff85d3c7c1d75c98f675a81642476af57b188d0298fc98d1"} Jun 03 16:46:39.693951 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:39.693908 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-2lgjn" podStartSLOduration=17.320873546 podStartE2EDuration="19.693895419s" podCreationTimestamp="2026-06-03 16:46:20 +0000 UTC" firstStartedPulling="2026-06-03 16:46:36.579932883 +0000 UTC m=+168.963280279" lastFinishedPulling="2026-06-03 16:46:38.952954751 +0000 UTC m=+171.336302152" observedRunningTime="2026-06-03 16:46:39.693268902 +0000 UTC m=+172.076616321" watchObservedRunningTime="2026-06-03 16:46:39.693895419 +0000 UTC m=+172.077242880" Jun 03 16:46:39.710873 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:39.710821 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-f9zht" podStartSLOduration=17.16949368 podStartE2EDuration="19.71080948s" podCreationTimestamp="2026-06-03 16:46:20 +0000 UTC" firstStartedPulling="2026-06-03 16:46:36.41006011 +0000 UTC m=+168.793407510" lastFinishedPulling="2026-06-03 16:46:38.951375908 +0000 UTC m=+171.334723310" observedRunningTime="2026-06-03 16:46:39.71061567 +0000 UTC m=+172.093963083" watchObservedRunningTime="2026-06-03 16:46:39.71080948 +0000 UTC m=+172.094156896" Jun 03 16:46:49.555129 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.555091 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cqnsv"] Jun 03 16:46:49.559971 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.559955 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.566333 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.566252 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rc6mw\"" Jun 03 16:46:49.566652 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.566634 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 16:46:49.566722 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.566649 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 16:46:49.588981 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.588960 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cqnsv"] Jun 03 16:46:49.589911 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.589894 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm"] Jun 03 16:46:49.593053 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.593039 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.596341 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.596325 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v55pm\"" Jun 03 16:46:49.596649 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.596635 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 03 16:46:49.597049 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.597036 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 03 16:46:49.604603 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.604580 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8488d7768f-pwvnk"] Jun 03 16:46:49.606220 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.606201 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm"] Jun 03 16:46:49.619280 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.619368 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrsjx\" (UniqueName: \"kubernetes.io/projected/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-kube-api-access-rrsjx\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.619407 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c753777-850b-4a83-9523-aef26d98fda7-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-skmzm\" (UID: \"7c753777-850b-4a83-9523-aef26d98fda7\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.619449 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c753777-850b-4a83-9523-aef26d98fda7-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-skmzm\" (UID: \"7c753777-850b-4a83-9523-aef26d98fda7\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.619449 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-crio-socket\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.619522 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-data-volume\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.619522 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.619496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.628952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.628933 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-869d9fc789-8rpgv"] Jun 03 16:46:49.631899 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.631885 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.649384 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.649366 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-869d9fc789-8rpgv"] Jun 03 16:46:49.720712 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720690 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.720834 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-image-registry-private-configuration\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.720834 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720761 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-ca-trust-extracted\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.720834 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720824 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.720962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720858 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cv8g\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-kube-api-access-9cv8g\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.720962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-registry-certificates\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.720962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720940 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrsjx\" (UniqueName: \"kubernetes.io/projected/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-kube-api-access-rrsjx\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.721088 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-trusted-ca\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.721088 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.720988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-installation-pull-secrets\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.721088 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721063 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c753777-850b-4a83-9523-aef26d98fda7-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-skmzm\" (UID: \"7c753777-850b-4a83-9523-aef26d98fda7\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.721236 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721095 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c753777-850b-4a83-9523-aef26d98fda7-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-skmzm\" (UID: \"7c753777-850b-4a83-9523-aef26d98fda7\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.721236 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-bound-sa-token\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.721236 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-registry-tls\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.721236 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721184 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-crio-socket\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.721236 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721208 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-data-volume\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.721439 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721278 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-crio-socket\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.721486 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721468 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.721545 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721487 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-data-volume\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.721825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.721804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c753777-850b-4a83-9523-aef26d98fda7-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-skmzm\" (UID: \"7c753777-850b-4a83-9523-aef26d98fda7\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.723167 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.723149 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.723451 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.723434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c753777-850b-4a83-9523-aef26d98fda7-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-skmzm\" (UID: \"7c753777-850b-4a83-9523-aef26d98fda7\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.733103 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.733075 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrsjx\" (UniqueName: \"kubernetes.io/projected/db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53-kube-api-access-rrsjx\") pod \"insights-runtime-extractor-cqnsv\" (UID: \"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53\") " pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.821784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.821729 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-trusted-ca\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.821784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.821757 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-installation-pull-secrets\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.821930 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.821909 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-bound-sa-token\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.821978 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.821954 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-registry-tls\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822038 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822023 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-image-registry-private-configuration\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822246 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822227 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-ca-trust-extracted\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822307 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822279 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9cv8g\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-kube-api-access-9cv8g\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822362 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822308 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-registry-certificates\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822672 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822629 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-ca-trust-extracted\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822749 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822695 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-trusted-ca\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.822970 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.822955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-registry-certificates\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.824233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.824202 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-installation-pull-secrets\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.824407 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.824388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-registry-tls\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.824517 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.824501 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-image-registry-private-configuration\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.838077 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.838056 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cv8g\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-kube-api-access-9cv8g\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.838374 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.838358 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eaa859a3-7adb-407e-8c0c-8c8bc4a10070-bound-sa-token\") pod \"image-registry-869d9fc789-8rpgv\" (UID: \"eaa859a3-7adb-407e-8c0c-8c8bc4a10070\") " pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:49.869801 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.869783 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cqnsv" Jun 03 16:46:49.900973 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.900952 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" Jun 03 16:46:49.939617 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:49.939589 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:50.056765 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.056737 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cqnsv"] Jun 03 16:46:50.060868 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:50.060839 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3d34e3_09e2_47d7_ae3d_50f9a2a1ee53.slice/crio-34f0852093f1992da0bc7c21149cb51cefdfc99a6d3c5e7262a7256ada32b897 WatchSource:0}: Error finding container 34f0852093f1992da0bc7c21149cb51cefdfc99a6d3c5e7262a7256ada32b897: Status 404 returned error can't find the container with id 34f0852093f1992da0bc7c21149cb51cefdfc99a6d3c5e7262a7256ada32b897 Jun 03 16:46:50.090763 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.090742 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm"] Jun 03 16:46:50.094126 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:50.094099 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c753777_850b_4a83_9523_aef26d98fda7.slice/crio-f3431a44d76618f4e5a676c5220091678a0b4eeec5ec17056a175c413755ea38 WatchSource:0}: Error finding container f3431a44d76618f4e5a676c5220091678a0b4eeec5ec17056a175c413755ea38: Status 404 returned error can't find the container with id f3431a44d76618f4e5a676c5220091678a0b4eeec5ec17056a175c413755ea38 Jun 03 16:46:50.127044 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.127011 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-869d9fc789-8rpgv"] Jun 03 16:46:50.129891 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:50.129864 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa859a3_7adb_407e_8c0c_8c8bc4a10070.slice/crio-d121124ee6c936135df14367d5d844256c4a49f934c046749d461aab1b71d615 WatchSource:0}: Error finding container d121124ee6c936135df14367d5d844256c4a49f934c046749d461aab1b71d615: Status 404 returned error can't find the container with id d121124ee6c936135df14367d5d844256c4a49f934c046749d461aab1b71d615 Jun 03 16:46:50.706455 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.706418 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" event={"ID":"eaa859a3-7adb-407e-8c0c-8c8bc4a10070","Type":"ContainerStarted","Data":"c3ef8e2060d85a8e80c9ba403c596b2a6f281d32ffb5a1f433e3824edaa18ca0"} Jun 03 16:46:50.706455 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.706460 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" event={"ID":"eaa859a3-7adb-407e-8c0c-8c8bc4a10070","Type":"ContainerStarted","Data":"d121124ee6c936135df14367d5d844256c4a49f934c046749d461aab1b71d615"} Jun 03 16:46:50.706984 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.706583 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:46:50.708303 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.708263 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqnsv" event={"ID":"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53","Type":"ContainerStarted","Data":"8cf2f34ef984376e095a31d2ad1f1abf4aad7e491ae90b52125beaf7b9d96db0"} Jun 03 16:46:50.708429 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.708311 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqnsv" event={"ID":"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53","Type":"ContainerStarted","Data":"34f0852093f1992da0bc7c21149cb51cefdfc99a6d3c5e7262a7256ada32b897"} Jun 03 16:46:50.709574 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.709534 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" event={"ID":"7c753777-850b-4a83-9523-aef26d98fda7","Type":"ContainerStarted","Data":"f3431a44d76618f4e5a676c5220091678a0b4eeec5ec17056a175c413755ea38"} Jun 03 16:46:50.733666 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:50.733625 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podStartSLOduration=1.733612709 podStartE2EDuration="1.733612709s" podCreationTimestamp="2026-06-03 16:46:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:46:50.733115216 +0000 UTC m=+183.116462639" watchObservedRunningTime="2026-06-03 16:46:50.733612709 +0000 UTC m=+183.116960131" Jun 03 16:46:51.714530 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:51.714479 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" event={"ID":"7c753777-850b-4a83-9523-aef26d98fda7","Type":"ContainerStarted","Data":"8203147cdf3133bd11a417e5b91d60569375910c31a156453591eb3c673968ed"} Jun 03 16:46:51.716349 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:51.716319 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqnsv" event={"ID":"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53","Type":"ContainerStarted","Data":"362ba0e59452bb267bc6fc78e5721b864b131426a054a5e27b3030d6789fe340"} Jun 03 16:46:52.720866 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:52.720838 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqnsv" event={"ID":"db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53","Type":"ContainerStarted","Data":"8f18343dfb0c3afca680caceed5547b402c67db77a4d3eed73d3020c1097f210"} Jun 03 16:46:52.736455 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:52.736412 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cqnsv" podStartSLOduration=1.242624541 podStartE2EDuration="3.73640014s" podCreationTimestamp="2026-06-03 16:46:49 +0000 UTC" firstStartedPulling="2026-06-03 16:46:50.113551081 +0000 UTC m=+182.496898478" lastFinishedPulling="2026-06-03 16:46:52.60732668 +0000 UTC m=+184.990674077" observedRunningTime="2026-06-03 16:46:52.736358792 +0000 UTC m=+185.119706210" watchObservedRunningTime="2026-06-03 16:46:52.73640014 +0000 UTC m=+185.119747558" Jun 03 16:46:52.736901 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:52.736865 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-skmzm" podStartSLOduration=2.318635531 podStartE2EDuration="3.736854942s" podCreationTimestamp="2026-06-03 16:46:49 +0000 UTC" firstStartedPulling="2026-06-03 16:46:50.095959893 +0000 UTC m=+182.479307290" lastFinishedPulling="2026-06-03 16:46:51.514179303 +0000 UTC m=+183.897526701" observedRunningTime="2026-06-03 16:46:51.730154717 +0000 UTC m=+184.113502139" watchObservedRunningTime="2026-06-03 16:46:52.736854942 +0000 UTC m=+185.120202429" Jun 03 16:46:53.204162 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.204130 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-565988f656-68llw"] Jun 03 16:46:53.207292 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.207276 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.209773 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.209743 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 03 16:46:53.209894 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.209775 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 03 16:46:53.210258 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.210231 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 03 16:46:53.210334 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.210262 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7hgwb\"" Jun 03 16:46:53.210382 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.210342 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 03 16:46:53.210382 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.210342 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 03 16:46:53.210479 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.210392 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 03 16:46:53.210479 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.210361 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 03 16:46:53.217283 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.217262 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565988f656-68llw"] Jun 03 16:46:53.258638 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.258614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-config\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.258738 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.258662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-oauth-config\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.258738 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.258681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-oauth-serving-cert\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.258814 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.258738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqnnq\" (UniqueName: \"kubernetes.io/projected/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-kube-api-access-tqnnq\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.258814 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.258795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-service-ca\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.258910 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.258815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-serving-cert\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.359439 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.359414 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-config\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.359531 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.359454 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-oauth-config\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.359531 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.359471 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-oauth-serving-cert\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.359531 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.359493 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tqnnq\" (UniqueName: \"kubernetes.io/projected/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-kube-api-access-tqnnq\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.359531 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.359530 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-service-ca\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.359754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.359578 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-serving-cert\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.360217 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.360198 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-oauth-serving-cert\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.360318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.360219 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-service-ca\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.360318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.360224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-config\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.361944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.361924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-serving-cert\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.362017 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.361931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-oauth-config\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.367379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.367360 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqnnq\" (UniqueName: \"kubernetes.io/projected/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-kube-api-access-tqnnq\") pod \"console-565988f656-68llw\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.516310 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.516253 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565988f656-68llw" Jun 03 16:46:53.637066 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.637040 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565988f656-68llw"] Jun 03 16:46:53.640458 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:53.640434 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55da7cf8_c6c6_44dc_8615_79fd6ff0e36c.slice/crio-5e9fb4037e737391dde0a04d359cfe6f1bfa877cd5e60a5d1bf391f39db2fcf5 WatchSource:0}: Error finding container 5e9fb4037e737391dde0a04d359cfe6f1bfa877cd5e60a5d1bf391f39db2fcf5: Status 404 returned error can't find the container with id 5e9fb4037e737391dde0a04d359cfe6f1bfa877cd5e60a5d1bf391f39db2fcf5 Jun 03 16:46:53.728474 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:53.728437 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565988f656-68llw" event={"ID":"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c","Type":"ContainerStarted","Data":"5e9fb4037e737391dde0a04d359cfe6f1bfa877cd5e60a5d1bf391f39db2fcf5"} Jun 03 16:46:56.917635 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.917600 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g"] Jun 03 16:46:56.922655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.922632 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.925205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.925152 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 03 16:46:56.925205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.925173 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 03 16:46:56.925205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.925153 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 16:46:56.925205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.925206 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5t5fb\"" Jun 03 16:46:56.925459 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.925153 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 03 16:46:56.931659 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.930525 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fj64g"] Jun 03 16:46:56.934840 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.934820 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g"] Jun 03 16:46:56.934951 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.934922 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.936896 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.936876 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 16:46:56.937186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.937167 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f7b2x\"" Jun 03 16:46:56.937625 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.937610 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 16:46:56.938308 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.938289 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 16:46:56.990807 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.990785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-wtmp\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.990931 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.990838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45ab17aa-36a7-4214-a570-bca699c6ce5b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.990931 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.990864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9vb\" (UniqueName: \"kubernetes.io/projected/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-api-access-zj9vb\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.990931 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.990893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.990931 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.990923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.991145 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-textfile\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.991145 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-sys\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.991145 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91b99905-fbd8-4832-a92e-b6ba2eef61ec-metrics-client-ca\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.991145 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.991400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991148 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-tls\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.991400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45ab17aa-36a7-4214-a570-bca699c6ce5b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.991400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-root\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.991400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4fnm\" (UniqueName: \"kubernetes.io/projected/91b99905-fbd8-4832-a92e-b6ba2eef61ec-kube-api-access-c4fnm\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:56.991400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:56.991400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:56.991385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.092709 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092678 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91b99905-fbd8-4832-a92e-b6ba2eef61ec-metrics-client-ca\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.092863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092720 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.092863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092753 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-tls\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.092863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092792 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45ab17aa-36a7-4214-a570-bca699c6ce5b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.092863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092816 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-root\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.092863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092841 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4fnm\" (UniqueName: \"kubernetes.io/projected/91b99905-fbd8-4832-a92e-b6ba2eef61ec-kube-api-access-c4fnm\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092891 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.093102 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:57.092929 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 03 16:46:57.093102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092939 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-root\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093102 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:46:57.093000 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-tls podName:91b99905-fbd8-4832-a92e-b6ba2eef61ec nodeName:}" failed. No retries permitted until 2026-06-03 16:46:57.592978766 +0000 UTC m=+189.976326167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-tls") pod "node-exporter-fj64g" (UID: "91b99905-fbd8-4832-a92e-b6ba2eef61ec") : secret "node-exporter-tls" not found Jun 03 16:46:57.093102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.092932 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093058 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-wtmp\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093107 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45ab17aa-36a7-4214-a570-bca699c6ce5b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093131 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9vb\" (UniqueName: \"kubernetes.io/projected/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-api-access-zj9vb\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093158 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093202 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093252 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-textfile\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093281 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-sys\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093329 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91b99905-fbd8-4832-a92e-b6ba2eef61ec-metrics-client-ca\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093370 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-sys\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093428 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.093810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093505 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45ab17aa-36a7-4214-a570-bca699c6ce5b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.093810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093591 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-wtmp\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093650 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-textfile\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.093810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.093778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.094155 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.094130 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45ab17aa-36a7-4214-a570-bca699c6ce5b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.095904 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.095874 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.096259 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.096238 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.096435 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.096416 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.109148 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.109124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4fnm\" (UniqueName: \"kubernetes.io/projected/91b99905-fbd8-4832-a92e-b6ba2eef61ec-kube-api-access-c4fnm\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.109998 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.109964 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9vb\" (UniqueName: \"kubernetes.io/projected/45ab17aa-36a7-4214-a570-bca699c6ce5b-kube-api-access-zj9vb\") pod \"kube-state-metrics-8469c486bb-vfj4g\" (UID: \"45ab17aa-36a7-4214-a570-bca699c6ce5b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.235530 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.235454 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" Jun 03 16:46:57.598604 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.598542 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-tls\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.600922 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.600892 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91b99905-fbd8-4832-a92e-b6ba2eef61ec-node-exporter-tls\") pod \"node-exporter-fj64g\" (UID: \"91b99905-fbd8-4832-a92e-b6ba2eef61ec\") " pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.844590 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:57.844546 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fj64g" Jun 03 16:46:57.943332 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:57.943294 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b99905_fbd8_4832_a92e_b6ba2eef61ec.slice/crio-45a6c081b5bc50cd9f277d593691eb859f9d80eb6b541b1f53a679a503f0a808 WatchSource:0}: Error finding container 45a6c081b5bc50cd9f277d593691eb859f9d80eb6b541b1f53a679a503f0a808: Status 404 returned error can't find the container with id 45a6c081b5bc50cd9f277d593691eb859f9d80eb6b541b1f53a679a503f0a808 Jun 03 16:46:58.061635 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:58.061611 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g"] Jun 03 16:46:58.064908 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:46:58.064882 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ab17aa_36a7_4214_a570_bca699c6ce5b.slice/crio-364d28058fc3ab33ed0f56d9bf5f80f4c9dc1445cddd0ca85129787337fddb16 WatchSource:0}: Error finding container 364d28058fc3ab33ed0f56d9bf5f80f4c9dc1445cddd0ca85129787337fddb16: Status 404 returned error can't find the container with id 364d28058fc3ab33ed0f56d9bf5f80f4c9dc1445cddd0ca85129787337fddb16 Jun 03 16:46:58.744854 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:58.744817 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565988f656-68llw" event={"ID":"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c","Type":"ContainerStarted","Data":"37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099"} Jun 03 16:46:58.747134 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:58.747097 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" event={"ID":"45ab17aa-36a7-4214-a570-bca699c6ce5b","Type":"ContainerStarted","Data":"364d28058fc3ab33ed0f56d9bf5f80f4c9dc1445cddd0ca85129787337fddb16"} Jun 03 16:46:58.749033 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:58.749003 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj64g" event={"ID":"91b99905-fbd8-4832-a92e-b6ba2eef61ec","Type":"ContainerStarted","Data":"45a6c081b5bc50cd9f277d593691eb859f9d80eb6b541b1f53a679a503f0a808"} Jun 03 16:46:58.766497 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:58.766199 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-565988f656-68llw" podStartSLOduration=1.412150017 podStartE2EDuration="5.766183783s" podCreationTimestamp="2026-06-03 16:46:53 +0000 UTC" firstStartedPulling="2026-06-03 16:46:53.642281781 +0000 UTC m=+186.025629181" lastFinishedPulling="2026-06-03 16:46:57.996315536 +0000 UTC m=+190.379662947" observedRunningTime="2026-06-03 16:46:58.763953846 +0000 UTC m=+191.147301268" watchObservedRunningTime="2026-06-03 16:46:58.766183783 +0000 UTC m=+191.149531203" Jun 03 16:46:59.610279 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.610236 2578 patch_prober.go:28] interesting pod/image-registry-8488d7768f-pwvnk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:46:59.610693 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.610293 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:46:59.753514 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.753481 2578 generic.go:358] "Generic (PLEG): container finished" podID="91b99905-fbd8-4832-a92e-b6ba2eef61ec" containerID="da4e438f477f77cae635b207d1a8ef340e824acbb4879c89496a2105247a2fce" exitCode=0 Jun 03 16:46:59.753688 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.753579 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj64g" event={"ID":"91b99905-fbd8-4832-a92e-b6ba2eef61ec","Type":"ContainerDied","Data":"da4e438f477f77cae635b207d1a8ef340e824acbb4879c89496a2105247a2fce"} Jun 03 16:46:59.755607 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.755583 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" event={"ID":"45ab17aa-36a7-4214-a570-bca699c6ce5b","Type":"ContainerStarted","Data":"210ad84eb95ead969096f79bc089f44b3dcaf447b01eb8e0bec9e0e25d5f6756"} Jun 03 16:46:59.755702 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.755611 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" event={"ID":"45ab17aa-36a7-4214-a570-bca699c6ce5b","Type":"ContainerStarted","Data":"c8da554c8a5162281114ae8df6135faf397f6aef8ee9472d69b08640408b447c"} Jun 03 16:46:59.755702 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.755624 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" event={"ID":"45ab17aa-36a7-4214-a570-bca699c6ce5b","Type":"ContainerStarted","Data":"3ea487290df877641830630d5fd4c72fd8437793529ac012d533a8b4259c805e"} Jun 03 16:46:59.789149 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.789105 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-vfj4g" podStartSLOduration=2.536478175 podStartE2EDuration="3.78908911s" podCreationTimestamp="2026-06-03 16:46:56 +0000 UTC" firstStartedPulling="2026-06-03 16:46:58.066956982 +0000 UTC m=+190.450304392" lastFinishedPulling="2026-06-03 16:46:59.319567905 +0000 UTC m=+191.702915327" observedRunningTime="2026-06-03 16:46:59.787580757 +0000 UTC m=+192.170928177" watchObservedRunningTime="2026-06-03 16:46:59.78908911 +0000 UTC m=+192.172436530" Jun 03 16:46:59.990358 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.990277 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b64457c-qrlft"] Jun 03 16:46:59.993965 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.993943 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:46:59.996190 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996164 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 03 16:46:59.996318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996200 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 03 16:46:59.996318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996216 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2e519s6jpb0p\"" Jun 03 16:46:59.996318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996236 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qq79z\"" Jun 03 16:46:59.996318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996281 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 03 16:46:59.996546 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996528 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 03 16:46:59.996607 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:46:59.996545 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 03 16:47:00.004645 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.004628 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b64457c-qrlft"] Jun 03 16:47:00.122428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122400 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-grpc-tls\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41057247-908a-4105-b6f1-bb08a0de923e-metrics-client-ca\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7s8p\" (UniqueName: \"kubernetes.io/projected/41057247-908a-4105-b6f1-bb08a0de923e-kube-api-access-q7s8p\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.122758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.122729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-tls\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.223712 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.223683 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41057247-908a-4105-b6f1-bb08a0de923e-metrics-client-ca\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.223876 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.223718 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.223876 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.223822 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.224011 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.223989 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q7s8p\" (UniqueName: \"kubernetes.io/projected/41057247-908a-4105-b6f1-bb08a0de923e-kube-api-access-q7s8p\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.224297 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.224250 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.224431 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.224315 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-tls\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.224431 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.224404 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.224811 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.224433 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-grpc-tls\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.224914 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.224499 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41057247-908a-4105-b6f1-bb08a0de923e-metrics-client-ca\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.226553 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.226523 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.226767 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.226735 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.226940 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.226923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.227001 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.226946 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-grpc-tls\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.227109 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.227088 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.227145 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.227100 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/41057247-908a-4105-b6f1-bb08a0de923e-secret-thanos-querier-tls\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.236647 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.236630 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7s8p\" (UniqueName: \"kubernetes.io/projected/41057247-908a-4105-b6f1-bb08a0de923e-kube-api-access-q7s8p\") pod \"thanos-querier-b64457c-qrlft\" (UID: \"41057247-908a-4105-b6f1-bb08a0de923e\") " pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.302681 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.302617 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:00.426167 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.426137 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b64457c-qrlft"] Jun 03 16:47:00.428968 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:47:00.428941 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41057247_908a_4105_b6f1_bb08a0de923e.slice/crio-d2b5b73cce1e5390f4ff01661238cf12309d0cd152140b38409d5b1363b34f1e WatchSource:0}: Error finding container d2b5b73cce1e5390f4ff01661238cf12309d0cd152140b38409d5b1363b34f1e: Status 404 returned error can't find the container with id d2b5b73cce1e5390f4ff01661238cf12309d0cd152140b38409d5b1363b34f1e Jun 03 16:47:00.759814 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.759783 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj64g" event={"ID":"91b99905-fbd8-4832-a92e-b6ba2eef61ec","Type":"ContainerStarted","Data":"38c261644dd0ff273eb8af0bd800e704aaa1a1848804619b634a70f3579649b5"} Jun 03 16:47:00.760211 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.759821 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj64g" event={"ID":"91b99905-fbd8-4832-a92e-b6ba2eef61ec","Type":"ContainerStarted","Data":"62f4240e111bad2f88e6e9f158d8354fde150161058592a88f93ff08b599d7e3"} Jun 03 16:47:00.760933 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.760909 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"d2b5b73cce1e5390f4ff01661238cf12309d0cd152140b38409d5b1363b34f1e"} Jun 03 16:47:00.777366 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:00.777311 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fj64g" podStartSLOduration=3.4072795080000002 podStartE2EDuration="4.777296907s" podCreationTimestamp="2026-06-03 16:46:56 +0000 UTC" firstStartedPulling="2026-06-03 16:46:57.947620794 +0000 UTC m=+190.330968191" lastFinishedPulling="2026-06-03 16:46:59.317638184 +0000 UTC m=+191.700985590" observedRunningTime="2026-06-03 16:47:00.776780653 +0000 UTC m=+193.160128071" watchObservedRunningTime="2026-06-03 16:47:00.777296907 +0000 UTC m=+193.160644327" Jun 03 16:47:01.409496 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.409465 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86d49fbb7c-5sb9v"] Jun 03 16:47:01.412819 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.412793 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.415058 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.415032 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 03 16:47:01.415233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.415057 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zkqc6\"" Jun 03 16:47:01.415233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.415097 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 03 16:47:01.415233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.415110 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f9a2cc2n7291b\"" Jun 03 16:47:01.415233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.415057 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 03 16:47:01.415491 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.415382 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 03 16:47:01.420588 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.420547 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86d49fbb7c-5sb9v"] Jun 03 16:47:01.535481 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1e34a595-64d4-4375-a219-cd674a3406c3-audit-log\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.535646 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-secret-metrics-server-client-certs\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.535646 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535527 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-client-ca-bundle\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.535646 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-secret-metrics-server-tls\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.535646 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535628 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e34a595-64d4-4375-a219-cd674a3406c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.535646 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1e34a595-64d4-4375-a219-cd674a3406c3-metrics-server-audit-profiles\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.535832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.535713 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slfd7\" (UniqueName: \"kubernetes.io/projected/1e34a595-64d4-4375-a219-cd674a3406c3-kube-api-access-slfd7\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636516 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636475 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e34a595-64d4-4375-a219-cd674a3406c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636690 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636521 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1e34a595-64d4-4375-a219-cd674a3406c3-metrics-server-audit-profiles\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636690 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636603 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slfd7\" (UniqueName: \"kubernetes.io/projected/1e34a595-64d4-4375-a219-cd674a3406c3-kube-api-access-slfd7\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636690 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636631 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1e34a595-64d4-4375-a219-cd674a3406c3-audit-log\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636690 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636687 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-secret-metrics-server-client-certs\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636961 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636713 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-client-ca-bundle\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.636961 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.636741 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-secret-metrics-server-tls\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.637272 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.637233 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e34a595-64d4-4375-a219-cd674a3406c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.637962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.637939 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1e34a595-64d4-4375-a219-cd674a3406c3-audit-log\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.638469 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.638443 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1e34a595-64d4-4375-a219-cd674a3406c3-metrics-server-audit-profiles\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.639748 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.639726 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-secret-metrics-server-tls\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.639831 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.639815 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-secret-metrics-server-client-certs\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.640382 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.640325 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e34a595-64d4-4375-a219-cd674a3406c3-client-ca-bundle\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.645708 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.645684 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slfd7\" (UniqueName: \"kubernetes.io/projected/1e34a595-64d4-4375-a219-cd674a3406c3-kube-api-access-slfd7\") pod \"metrics-server-86d49fbb7c-5sb9v\" (UID: \"1e34a595-64d4-4375-a219-cd674a3406c3\") " pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.723841 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.723776 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:01.842575 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:01.842525 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86d49fbb7c-5sb9v"] Jun 03 16:47:01.845468 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:47:01.845431 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e34a595_64d4_4375_a219_cd674a3406c3.slice/crio-714511c1cf25a7efcf2e335378dabbabcfa4184ad86af2ee91320f3a78b58823 WatchSource:0}: Error finding container 714511c1cf25a7efcf2e335378dabbabcfa4184ad86af2ee91320f3a78b58823: Status 404 returned error can't find the container with id 714511c1cf25a7efcf2e335378dabbabcfa4184ad86af2ee91320f3a78b58823 Jun 03 16:47:02.407632 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.407597 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-84958bf845-xm265"] Jun 03 16:47:02.411202 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.411170 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.420444 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.420225 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 03 16:47:02.433695 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.433657 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84958bf845-xm265"] Jun 03 16:47:02.544387 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkc96\" (UniqueName: \"kubernetes.io/projected/7c750956-f508-4c4a-8733-cb47c77fd710-kube-api-access-zkc96\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.544542 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-console-config\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.544542 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-serving-cert\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.544632 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544579 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-trusted-ca-bundle\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.544632 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-service-ca\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.544702 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-oauth-config\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.544735 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.544719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-oauth-serving-cert\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.645746 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.645716 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-console-config\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.645840 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.645790 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-serving-cert\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.645892 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.645847 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-trusted-ca-bundle\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.645953 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.645889 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-service-ca\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.645953 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.645942 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-oauth-config\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.646072 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.645997 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-oauth-serving-cert\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.646072 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.646040 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zkc96\" (UniqueName: \"kubernetes.io/projected/7c750956-f508-4c4a-8733-cb47c77fd710-kube-api-access-zkc96\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.646341 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.646315 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-console-config\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.646899 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.646862 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-service-ca\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.646995 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.646904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-oauth-serving-cert\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.647070 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.647022 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-trusted-ca-bundle\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.649080 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.649050 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-serving-cert\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.649080 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.649055 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-oauth-config\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.655884 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.655862 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkc96\" (UniqueName: \"kubernetes.io/projected/7c750956-f508-4c4a-8733-cb47c77fd710-kube-api-access-zkc96\") pod \"console-84958bf845-xm265\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.723265 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.723220 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:02.770754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.770368 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"1e4aa42a5eccad9b486fc14fa1dfb578159ce3de02bfb54c1c75c187db8edc97"} Jun 03 16:47:02.770754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.770407 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"b04f1f97c473d1a1bfff26e3cd960cb7f9c4a2a63e13dd836e30ab52374752d5"} Jun 03 16:47:02.770754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.770423 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"015be637846a04dca4e14b6af436606e019b57dac8a86c3974e80acae2038974"} Jun 03 16:47:02.772048 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.772019 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" event={"ID":"1e34a595-64d4-4375-a219-cd674a3406c3","Type":"ContainerStarted","Data":"714511c1cf25a7efcf2e335378dabbabcfa4184ad86af2ee91320f3a78b58823"} Jun 03 16:47:02.870644 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:02.870616 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84958bf845-xm265"] Jun 03 16:47:02.873398 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:47:02.873370 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c750956_f508_4c4a_8733_cb47c77fd710.slice/crio-ecdf4f62e368fe668540f6e0fabcfea699a11ab8cec44dae42703a5816eb7583 WatchSource:0}: Error finding container ecdf4f62e368fe668540f6e0fabcfea699a11ab8cec44dae42703a5816eb7583: Status 404 returned error can't find the container with id ecdf4f62e368fe668540f6e0fabcfea699a11ab8cec44dae42703a5816eb7583 Jun 03 16:47:03.517203 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.517163 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-565988f656-68llw" Jun 03 16:47:03.517377 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.517218 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-565988f656-68llw" Jun 03 16:47:03.518783 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.518760 2578 patch_prober.go:28] interesting pod/console-565988f656-68llw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body= Jun 03 16:47:03.518908 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.518803 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-565988f656-68llw" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerName="console" probeResult="failure" output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" Jun 03 16:47:03.777846 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.777770 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84958bf845-xm265" event={"ID":"7c750956-f508-4c4a-8733-cb47c77fd710","Type":"ContainerStarted","Data":"1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47"} Jun 03 16:47:03.777846 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.777814 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84958bf845-xm265" event={"ID":"7c750956-f508-4c4a-8733-cb47c77fd710","Type":"ContainerStarted","Data":"ecdf4f62e368fe668540f6e0fabcfea699a11ab8cec44dae42703a5816eb7583"} Jun 03 16:47:03.805674 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:03.805633 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-84958bf845-xm265" podStartSLOduration=1.805620154 podStartE2EDuration="1.805620154s" podCreationTimestamp="2026-06-03 16:47:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:47:03.805440904 +0000 UTC m=+196.188788326" watchObservedRunningTime="2026-06-03 16:47:03.805620154 +0000 UTC m=+196.188967573" Jun 03 16:47:04.783967 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.783933 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"9de7739d212eae3d57014b5bf655b189487a682f3dcb3a86c89ce930fdf23706"} Jun 03 16:47:04.784391 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.783975 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"96b85478caa630121e46af75aaea1653034130b6b45070efd207ea8417202332"} Jun 03 16:47:04.784391 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.783989 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" event={"ID":"41057247-908a-4105-b6f1-bb08a0de923e","Type":"ContainerStarted","Data":"e34d3eec22c27cfe6d67228b5d8eeec3586506e0624735c350a49f8b2a4f0b2c"} Jun 03 16:47:04.784391 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.784095 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:04.785189 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.785166 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" event={"ID":"1e34a595-64d4-4375-a219-cd674a3406c3","Type":"ContainerStarted","Data":"412f0c2d408fc0c2dd8a833f30f993efa6158afd36fd313b5d12ccb307234ee9"} Jun 03 16:47:04.812741 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.812700 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" podStartSLOduration=2.412922223 podStartE2EDuration="5.812687215s" podCreationTimestamp="2026-06-03 16:46:59 +0000 UTC" firstStartedPulling="2026-06-03 16:47:00.435470175 +0000 UTC m=+192.818817572" lastFinishedPulling="2026-06-03 16:47:03.835235167 +0000 UTC m=+196.218582564" observedRunningTime="2026-06-03 16:47:04.81190362 +0000 UTC m=+197.195251038" watchObservedRunningTime="2026-06-03 16:47:04.812687215 +0000 UTC m=+197.196034633" Jun 03 16:47:04.831689 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:04.831651 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" podStartSLOduration=1.845314857 podStartE2EDuration="3.831636482s" podCreationTimestamp="2026-06-03 16:47:01 +0000 UTC" firstStartedPulling="2026-06-03 16:47:01.847435525 +0000 UTC m=+194.230782922" lastFinishedPulling="2026-06-03 16:47:03.833757147 +0000 UTC m=+196.217104547" observedRunningTime="2026-06-03 16:47:04.830141559 +0000 UTC m=+197.213488978" watchObservedRunningTime="2026-06-03 16:47:04.831636482 +0000 UTC m=+197.214983903" Jun 03 16:47:09.609299 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:09.609262 2578 patch_prober.go:28] interesting pod/image-registry-8488d7768f-pwvnk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:09.609777 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:09.609318 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:09.943459 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:09.943388 2578 patch_prober.go:28] interesting pod/image-registry-869d9fc789-8rpgv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:09.943459 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:09.943433 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:10.794514 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:10.794487 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b64457c-qrlft" Jun 03 16:47:11.720948 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:11.720909 2578 patch_prober.go:28] interesting pod/image-registry-869d9fc789-8rpgv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:11.721118 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:11.720961 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:12.724286 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:12.724247 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:12.724286 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:12.724290 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:12.725488 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:12.725468 2578 patch_prober.go:28] interesting pod/console-84958bf845-xm265 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" start-of-body= Jun 03 16:47:12.725543 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:12.725510 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-84958bf845-xm265" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" containerName="console" probeResult="failure" output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" Jun 03 16:47:13.517657 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:13.517620 2578 patch_prober.go:28] interesting pod/console-565988f656-68llw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body= Jun 03 16:47:13.517808 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:13.517672 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-565988f656-68llw" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerName="console" probeResult="failure" output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" Jun 03 16:47:13.821100 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:13.821033 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565988f656-68llw"] Jun 03 16:47:13.849541 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:13.849510 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-645c8b888b-vws7l"] Jun 03 16:47:13.854999 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:13.854984 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:13.860634 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:13.860613 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645c8b888b-vws7l"] Jun 03 16:47:14.036764 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.036737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-service-ca\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.036908 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.036777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-serving-cert\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.036908 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.036796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsqnh\" (UniqueName: \"kubernetes.io/projected/f6501044-9e22-4216-821f-bae364c66f99-kube-api-access-rsqnh\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.036908 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.036873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-oauth-serving-cert\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.037005 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.036969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-oauth-config\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.037038 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.037026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-trusted-ca-bundle\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.037068 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.037050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-console-config\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.137806 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137774 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-serving-cert\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.137806 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137811 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqnh\" (UniqueName: \"kubernetes.io/projected/f6501044-9e22-4216-821f-bae364c66f99-kube-api-access-rsqnh\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137835 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-oauth-serving-cert\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137883 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-oauth-config\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137911 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-trusted-ca-bundle\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137930 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-console-config\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.137960 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-service-ca\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138683 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.138655 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-service-ca\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138806 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.138729 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-oauth-serving-cert\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138872 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.138841 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-console-config\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.138934 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.138848 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-trusted-ca-bundle\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.140432 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.140412 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-serving-cert\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.140509 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.140442 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-oauth-config\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.144994 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.144975 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqnh\" (UniqueName: \"kubernetes.io/projected/f6501044-9e22-4216-821f-bae364c66f99-kube-api-access-rsqnh\") pod \"console-645c8b888b-vws7l\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.165016 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.164994 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:14.285216 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.285191 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645c8b888b-vws7l"] Jun 03 16:47:14.287795 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:47:14.287755 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6501044_9e22_4216_821f_bae364c66f99.slice/crio-7b81c70f5babff9e86d63e373901605a5d9c97452edad73d118c871ea0e0b4fe WatchSource:0}: Error finding container 7b81c70f5babff9e86d63e373901605a5d9c97452edad73d118c871ea0e0b4fe: Status 404 returned error can't find the container with id 7b81c70f5babff9e86d63e373901605a5d9c97452edad73d118c871ea0e0b4fe Jun 03 16:47:14.624310 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.624258 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerName="registry" containerID="cri-o://3761ee2c3341b5ad56d49144e060e484c5fbba0171771c682e4715a724337eb3" gracePeriod=30 Jun 03 16:47:14.813412 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.813382 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c8b888b-vws7l" event={"ID":"f6501044-9e22-4216-821f-bae364c66f99","Type":"ContainerStarted","Data":"adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d"} Jun 03 16:47:14.813583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.813423 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c8b888b-vws7l" event={"ID":"f6501044-9e22-4216-821f-bae364c66f99","Type":"ContainerStarted","Data":"7b81c70f5babff9e86d63e373901605a5d9c97452edad73d118c871ea0e0b4fe"} Jun 03 16:47:14.814681 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.814657 2578 generic.go:358] "Generic (PLEG): container finished" podID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerID="3761ee2c3341b5ad56d49144e060e484c5fbba0171771c682e4715a724337eb3" exitCode=0 Jun 03 16:47:14.814775 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.814684 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" event={"ID":"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7","Type":"ContainerDied","Data":"3761ee2c3341b5ad56d49144e060e484c5fbba0171771c682e4715a724337eb3"} Jun 03 16:47:14.833102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.833055 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-645c8b888b-vws7l" podStartSLOduration=1.8330392739999999 podStartE2EDuration="1.833039274s" podCreationTimestamp="2026-06-03 16:47:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:47:14.830646064 +0000 UTC m=+207.213993484" watchObservedRunningTime="2026-06-03 16:47:14.833039274 +0000 UTC m=+207.216386695" Jun 03 16:47:14.862177 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:14.862160 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:47:15.046320 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046212 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-certificates\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046320 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046266 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-ca-trust-extracted\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046320 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046310 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046320 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046328 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-trusted-ca\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046668 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046356 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-bound-sa-token\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046668 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046383 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zw5t\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-kube-api-access-8zw5t\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046668 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046461 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-installation-pull-secrets\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046668 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046628 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-image-registry-private-configuration\") pod \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\" (UID: \"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7\") " Jun 03 16:47:15.046856 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046800 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:15.046856 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.046821 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:15.047046 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.047020 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-certificates\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.047106 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.047055 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-trusted-ca\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.048898 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.048868 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:47:15.049004 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.048896 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:47:15.049159 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.049140 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-kube-api-access-8zw5t" (OuterVolumeSpecName: "kube-api-access-8zw5t") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "kube-api-access-8zw5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:47:15.049217 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.049203 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:47:15.049266 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.049212 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:47:15.054936 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.054913 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" (UID: "1bc9511e-e38e-40eb-bd0e-8b038c20a4b7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:47:15.147476 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.147455 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-ca-trust-extracted\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.147581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.147479 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-registry-tls\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.147581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.147493 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-bound-sa-token\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.147581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.147505 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zw5t\" (UniqueName: \"kubernetes.io/projected/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-kube-api-access-8zw5t\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.147581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.147519 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-installation-pull-secrets\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.147581 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.147533 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7-image-registry-private-configuration\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:15.818810 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.818780 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" Jun 03 16:47:15.818970 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.818811 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8488d7768f-pwvnk" event={"ID":"1bc9511e-e38e-40eb-bd0e-8b038c20a4b7","Type":"ContainerDied","Data":"11336b97bcc7cc3392bb9040316db35d19b8b9909e8bbdeb9d2a05afd7de27ac"} Jun 03 16:47:15.818970 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.818855 2578 scope.go:117] "RemoveContainer" containerID="3761ee2c3341b5ad56d49144e060e484c5fbba0171771c682e4715a724337eb3" Jun 03 16:47:15.842381 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.842360 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8488d7768f-pwvnk"] Jun 03 16:47:15.848741 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:15.848722 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8488d7768f-pwvnk"] Jun 03 16:47:16.099301 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:16.099271 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" path="/var/lib/kubelet/pods/1bc9511e-e38e-40eb-bd0e-8b038c20a4b7/volumes" Jun 03 16:47:19.944368 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:19.944334 2578 patch_prober.go:28] interesting pod/image-registry-869d9fc789-8rpgv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:19.944730 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:19.944381 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:21.720061 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:21.720027 2578 patch_prober.go:28] interesting pod/image-registry-869d9fc789-8rpgv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:21.720446 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:21.720084 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:21.724182 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:21.724155 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:21.724312 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:21.724226 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:22.723790 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:22.723755 2578 patch_prober.go:28] interesting pod/console-84958bf845-xm265 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" start-of-body= Jun 03 16:47:22.724160 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:22.723802 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-84958bf845-xm265" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" containerName="console" probeResult="failure" output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" Jun 03 16:47:24.166149 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:24.166118 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:24.166520 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:24.166203 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:24.167532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:24.167514 2578 patch_prober.go:28] interesting pod/console-645c8b888b-vws7l container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused" start-of-body= Jun 03 16:47:24.167599 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:24.167551 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-645c8b888b-vws7l" podUID="f6501044-9e22-4216-821f-bae364c66f99" containerName="console" probeResult="failure" output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused" Jun 03 16:47:29.943507 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:29.943466 2578 patch_prober.go:28] interesting pod/image-registry-869d9fc789-8rpgv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:29.944006 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:29.943526 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:29.944006 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:29.943592 2578 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:47:29.944212 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:29.944175 2578 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"c3ef8e2060d85a8e80c9ba403c596b2a6f281d32ffb5a1f433e3824edaa18ca0"} pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" containerMessage="Container registry failed liveness probe, will be restarted" Jun 03 16:47:29.947446 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:29.947423 2578 patch_prober.go:28] interesting pod/image-registry-869d9fc789-8rpgv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:47:29.947575 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:29.947466 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:47:32.727975 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:32.727940 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:32.731787 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:32.731763 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:34.169500 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:34.169469 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:34.173369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:34.173332 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:47:34.229455 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:34.229424 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84958bf845-xm265"] Jun 03 16:47:34.873012 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:34.872981 2578 generic.go:358] "Generic (PLEG): container finished" podID="71f052c3-b28b-4eff-a502-518f583b7049" containerID="350ef60927fae0258cd220f406e62637fd1b3de6e23ddecc3dad038eaa9311a5" exitCode=0 Jun 03 16:47:34.873179 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:34.873053 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-5n57j" event={"ID":"71f052c3-b28b-4eff-a502-518f583b7049","Type":"ContainerDied","Data":"350ef60927fae0258cd220f406e62637fd1b3de6e23ddecc3dad038eaa9311a5"} Jun 03 16:47:34.873502 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:34.873486 2578 scope.go:117] "RemoveContainer" containerID="350ef60927fae0258cd220f406e62637fd1b3de6e23ddecc3dad038eaa9311a5" Jun 03 16:47:35.876722 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:35.876690 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-5n57j" event={"ID":"71f052c3-b28b-4eff-a502-518f583b7049","Type":"ContainerStarted","Data":"e5447e0ae99f0ec988920ace28c2fe1bc54a597ebbb2f927dc47e7da6a71a6f5"} Jun 03 16:47:36.493538 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:36.493510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmbrr_fa1a1e82-367d-49e7-9abb-6f49ef988e5d/dns-node-resolver/0.log" Jun 03 16:47:36.894623 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:36.894590 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-869d9fc789-8rpgv_eaa859a3-7adb-407e-8c0c-8c8bc4a10070/registry/0.log" Jun 03 16:47:37.493861 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:37.493839 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l74wk_06cc3bd3-c291-4ad8-8515-bb30440a138f/node-ca/0.log" Jun 03 16:47:38.841029 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:38.840991 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-565988f656-68llw" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerName="console" containerID="cri-o://37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099" gracePeriod=15 Jun 03 16:47:39.077332 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.077313 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-565988f656-68llw_55da7cf8-c6c6-44dc-8615-79fd6ff0e36c/console/0.log" Jun 03 16:47:39.077434 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.077369 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565988f656-68llw" Jun 03 16:47:39.140060 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.139996 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-oauth-config\") pod \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " Jun 03 16:47:39.140060 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140046 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-config\") pod \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " Jun 03 16:47:39.140232 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140126 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-service-ca\") pod \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " Jun 03 16:47:39.140232 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140152 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-oauth-serving-cert\") pod \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " Jun 03 16:47:39.140232 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140184 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqnnq\" (UniqueName: \"kubernetes.io/projected/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-kube-api-access-tqnnq\") pod \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " Jun 03 16:47:39.140232 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140220 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-serving-cert\") pod \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\" (UID: \"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c\") " Jun 03 16:47:39.140499 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140472 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-config" (OuterVolumeSpecName: "console-config") pod "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" (UID: "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:39.140574 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140527 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" (UID: "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:39.140627 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.140541 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-service-ca" (OuterVolumeSpecName: "service-ca") pod "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" (UID: "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:39.142324 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.142293 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" (UID: "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:47:39.142324 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.142316 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-kube-api-access-tqnnq" (OuterVolumeSpecName: "kube-api-access-tqnnq") pod "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" (UID: "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c"). InnerVolumeSpecName "kube-api-access-tqnnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:47:39.142324 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.142305 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" (UID: "55da7cf8-c6c6-44dc-8615-79fd6ff0e36c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:47:39.241553 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.241532 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-oauth-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:39.241553 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.241569 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:39.241701 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.241580 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-service-ca\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:39.241701 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.241589 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-oauth-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:39.241701 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.241599 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqnnq\" (UniqueName: \"kubernetes.io/projected/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-kube-api-access-tqnnq\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:39.241701 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.241609 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c-console-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:39.887819 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.887793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-565988f656-68llw_55da7cf8-c6c6-44dc-8615-79fd6ff0e36c/console/0.log" Jun 03 16:47:39.888228 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.887838 2578 generic.go:358] "Generic (PLEG): container finished" podID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerID="37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099" exitCode=2 Jun 03 16:47:39.888228 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.887895 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565988f656-68llw" event={"ID":"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c","Type":"ContainerDied","Data":"37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099"} Jun 03 16:47:39.888228 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.887931 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565988f656-68llw" Jun 03 16:47:39.888228 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.887958 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565988f656-68llw" event={"ID":"55da7cf8-c6c6-44dc-8615-79fd6ff0e36c","Type":"ContainerDied","Data":"5e9fb4037e737391dde0a04d359cfe6f1bfa877cd5e60a5d1bf391f39db2fcf5"} Jun 03 16:47:39.888228 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.887978 2578 scope.go:117] "RemoveContainer" containerID="37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099" Jun 03 16:47:39.897637 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.897618 2578 scope.go:117] "RemoveContainer" containerID="37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099" Jun 03 16:47:39.897918 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:47:39.897895 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099\": container with ID starting with 37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099 not found: ID does not exist" containerID="37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099" Jun 03 16:47:39.897992 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.897927 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099"} err="failed to get container status \"37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099\": rpc error: code = NotFound desc = could not find container \"37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099\": container with ID starting with 37040a70cbfa12422b7f519fb103fe33f2c219ebe308630029276b08dbe64099 not found: ID does not exist" Jun 03 16:47:39.908901 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.908880 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565988f656-68llw"] Jun 03 16:47:39.912660 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.912642 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-565988f656-68llw"] Jun 03 16:47:39.948102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:39.948086 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:47:40.101552 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:40.101522 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" path="/var/lib/kubelet/pods/55da7cf8-c6c6-44dc-8615-79fd6ff0e36c/volumes" Jun 03 16:47:41.729438 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:41.729409 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:41.733469 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:41.733446 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86d49fbb7c-5sb9v" Jun 03 16:47:49.922601 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:49.922545 2578 generic.go:358] "Generic (PLEG): container finished" podID="604f5077-f50a-4f66-8951-8ff2799b3e82" containerID="a1f5b7a8974f18c0116e21d5376367795fd650a6f941e3dd3058c806358364d6" exitCode=0 Jun 03 16:47:49.922942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:49.922626 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" event={"ID":"604f5077-f50a-4f66-8951-8ff2799b3e82","Type":"ContainerDied","Data":"a1f5b7a8974f18c0116e21d5376367795fd650a6f941e3dd3058c806358364d6"} Jun 03 16:47:49.922942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:49.922900 2578 scope.go:117] "RemoveContainer" containerID="a1f5b7a8974f18c0116e21d5376367795fd650a6f941e3dd3058c806358364d6" Jun 03 16:47:50.926586 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:50.926541 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-th5r7" event={"ID":"604f5077-f50a-4f66-8951-8ff2799b3e82","Type":"ContainerStarted","Data":"2f01f0f1e519e1ffe003591deb4649724a44606dcede8b38626146fbee295dc5"} Jun 03 16:47:54.962475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:54.962438 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" podUID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerName="registry" containerID="cri-o://c3ef8e2060d85a8e80c9ba403c596b2a6f281d32ffb5a1f433e3824edaa18ca0" gracePeriod=30 Jun 03 16:47:56.945680 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:56.945646 2578 generic.go:358] "Generic (PLEG): container finished" podID="eaa859a3-7adb-407e-8c0c-8c8bc4a10070" containerID="c3ef8e2060d85a8e80c9ba403c596b2a6f281d32ffb5a1f433e3824edaa18ca0" exitCode=0 Jun 03 16:47:56.946139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:56.945705 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" event={"ID":"eaa859a3-7adb-407e-8c0c-8c8bc4a10070","Type":"ContainerDied","Data":"c3ef8e2060d85a8e80c9ba403c596b2a6f281d32ffb5a1f433e3824edaa18ca0"} Jun 03 16:47:56.946139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:56.945727 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" event={"ID":"eaa859a3-7adb-407e-8c0c-8c8bc4a10070","Type":"ContainerStarted","Data":"be5aa092ae0f6498d9b89fd61ac04994b4056650178c99c8137d4ac094ff27c0"} Jun 03 16:47:56.946139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:56.945800 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:47:59.254576 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.254511 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-84958bf845-xm265" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" containerName="console" containerID="cri-o://1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47" gracePeriod=15 Jun 03 16:47:59.494653 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.494632 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84958bf845-xm265_7c750956-f508-4c4a-8733-cb47c77fd710/console/0.log" Jun 03 16:47:59.494747 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.494691 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:59.616497 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616471 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkc96\" (UniqueName: \"kubernetes.io/projected/7c750956-f508-4c4a-8733-cb47c77fd710-kube-api-access-zkc96\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.616641 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616511 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-console-config\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.616641 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616529 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-oauth-config\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.616641 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616575 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-oauth-serving-cert\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.616641 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616631 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-serving-cert\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.616832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616661 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-trusted-ca-bundle\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.616832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616687 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-service-ca\") pod \"7c750956-f508-4c4a-8733-cb47c77fd710\" (UID: \"7c750956-f508-4c4a-8733-cb47c77fd710\") " Jun 03 16:47:59.617029 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.616989 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-console-config" (OuterVolumeSpecName: "console-config") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:59.617138 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.617098 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:59.617207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.617151 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:59.617255 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.617204 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-service-ca" (OuterVolumeSpecName: "service-ca") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:47:59.618740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.618713 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:47:59.618740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.618729 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:47:59.618847 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.618737 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c750956-f508-4c4a-8733-cb47c77fd710-kube-api-access-zkc96" (OuterVolumeSpecName: "kube-api-access-zkc96") pod "7c750956-f508-4c4a-8733-cb47c77fd710" (UID: "7c750956-f508-4c4a-8733-cb47c77fd710"). InnerVolumeSpecName "kube-api-access-zkc96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:47:59.717334 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717301 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-console-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.717334 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717332 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-oauth-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.717334 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717342 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-oauth-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.717490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717352 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c750956-f508-4c4a-8733-cb47c77fd710-console-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.717490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717360 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-trusted-ca-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.717490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717369 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c750956-f508-4c4a-8733-cb47c77fd710-service-ca\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.717490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.717377 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkc96\" (UniqueName: \"kubernetes.io/projected/7c750956-f508-4c4a-8733-cb47c77fd710-kube-api-access-zkc96\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:47:59.954691 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.954631 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84958bf845-xm265_7c750956-f508-4c4a-8733-cb47c77fd710/console/0.log" Jun 03 16:47:59.954691 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.954670 2578 generic.go:358] "Generic (PLEG): container finished" podID="7c750956-f508-4c4a-8733-cb47c77fd710" containerID="1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47" exitCode=2 Jun 03 16:47:59.954852 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.954767 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84958bf845-xm265" Jun 03 16:47:59.954852 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.954764 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84958bf845-xm265" event={"ID":"7c750956-f508-4c4a-8733-cb47c77fd710","Type":"ContainerDied","Data":"1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47"} Jun 03 16:47:59.954852 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.954804 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84958bf845-xm265" event={"ID":"7c750956-f508-4c4a-8733-cb47c77fd710","Type":"ContainerDied","Data":"ecdf4f62e368fe668540f6e0fabcfea699a11ab8cec44dae42703a5816eb7583"} Jun 03 16:47:59.954852 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.954821 2578 scope.go:117] "RemoveContainer" containerID="1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47" Jun 03 16:47:59.963024 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.963004 2578 scope.go:117] "RemoveContainer" containerID="1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47" Jun 03 16:47:59.963251 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:47:59.963231 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47\": container with ID starting with 1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47 not found: ID does not exist" containerID="1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47" Jun 03 16:47:59.963310 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.963258 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47"} err="failed to get container status \"1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47\": rpc error: code = NotFound desc = could not find container \"1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47\": container with ID starting with 1b551bf663abea851a7c16c3adb434f221a1530f28dc8e69e51b790fc37a5d47 not found: ID does not exist" Jun 03 16:47:59.973832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.973807 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84958bf845-xm265"] Jun 03 16:47:59.979695 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:47:59.979677 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84958bf845-xm265"] Jun 03 16:48:00.019890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.019867 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:48:00.022017 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.021999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0cf9100-e9b8-42a0-93a8-f273610a87cf-metrics-certs\") pod \"network-metrics-daemon-rn64j\" (UID: \"e0cf9100-e9b8-42a0-93a8-f273610a87cf\") " pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:48:00.098379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.098351 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6fvvv\"" Jun 03 16:48:00.098628 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.098608 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" path="/var/lib/kubelet/pods/7c750956-f508-4c4a-8733-cb47c77fd710/volumes" Jun 03 16:48:00.106802 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.106773 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rn64j" Jun 03 16:48:00.235532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.235504 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rn64j"] Jun 03 16:48:00.237955 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:48:00.237928 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0cf9100_e9b8_42a0_93a8_f273610a87cf.slice/crio-d7ddf04c122c5d9466eb88e5229155e288547f0a51fe789e8c19bac0af6ba562 WatchSource:0}: Error finding container d7ddf04c122c5d9466eb88e5229155e288547f0a51fe789e8c19bac0af6ba562: Status 404 returned error can't find the container with id d7ddf04c122c5d9466eb88e5229155e288547f0a51fe789e8c19bac0af6ba562 Jun 03 16:48:00.959257 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:00.959221 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rn64j" event={"ID":"e0cf9100-e9b8-42a0-93a8-f273610a87cf","Type":"ContainerStarted","Data":"d7ddf04c122c5d9466eb88e5229155e288547f0a51fe789e8c19bac0af6ba562"} Jun 03 16:48:01.963604 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:01.963549 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rn64j" event={"ID":"e0cf9100-e9b8-42a0-93a8-f273610a87cf","Type":"ContainerStarted","Data":"f86c94d8a207aed21a93eced1d892c6fe48c00e432c0d122c156ff2214b99e9c"} Jun 03 16:48:01.963949 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:01.963610 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rn64j" event={"ID":"e0cf9100-e9b8-42a0-93a8-f273610a87cf","Type":"ContainerStarted","Data":"6740dba2080b39f862d32e7b4c6671287279077c8b8e732453f678863f6b8468"} Jun 03 16:48:01.978892 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:01.978418 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rn64j" podStartSLOduration=252.94504929 podStartE2EDuration="4m13.978401207s" podCreationTimestamp="2026-06-03 16:43:48 +0000 UTC" firstStartedPulling="2026-06-03 16:48:00.239725151 +0000 UTC m=+252.623072547" lastFinishedPulling="2026-06-03 16:48:01.273077062 +0000 UTC m=+253.656424464" observedRunningTime="2026-06-03 16:48:01.978277721 +0000 UTC m=+254.361625144" watchObservedRunningTime="2026-06-03 16:48:01.978401207 +0000 UTC m=+254.361748627" Jun 03 16:48:17.952758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:17.952724 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-869d9fc789-8rpgv" Jun 03 16:48:23.014611 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014581 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f5b447fc-mjdnm"] Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014887 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" containerName="console" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014898 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" containerName="console" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014911 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerName="registry" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014916 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerName="registry" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014922 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerName="console" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014928 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerName="console" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014974 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="1bc9511e-e38e-40eb-bd0e-8b038c20a4b7" containerName="registry" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014981 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="7c750956-f508-4c4a-8733-cb47c77fd710" containerName="console" Jun 03 16:48:23.014989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.014988 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="55da7cf8-c6c6-44dc-8615-79fd6ff0e36c" containerName="console" Jun 03 16:48:23.018897 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.018878 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.023690 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.023659 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f5b447fc-mjdnm"] Jun 03 16:48:23.097991 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.097959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-oauth-serving-cert\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.098130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.098005 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-config\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.098130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.098072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-oauth-config\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.098130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.098106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-service-ca\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.098264 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.098170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-trusted-ca-bundle\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.098264 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.098186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fblsn\" (UniqueName: \"kubernetes.io/projected/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-kube-api-access-fblsn\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.098264 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.098214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-serving-cert\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.198975 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.198937 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-serving-cert\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199094 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.198993 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-oauth-serving-cert\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199094 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199036 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-config\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199201 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199104 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-oauth-config\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199201 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199133 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-service-ca\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199201 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199183 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-trusted-ca-bundle\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199347 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199206 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fblsn\" (UniqueName: \"kubernetes.io/projected/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-kube-api-access-fblsn\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199836 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199808 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-config\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.199944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199922 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-service-ca\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.200012 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.199996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-trusted-ca-bundle\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.200246 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.200227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-oauth-serving-cert\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.201342 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.201322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-serving-cert\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.201463 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.201445 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-oauth-config\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.207125 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.207100 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fblsn\" (UniqueName: \"kubernetes.io/projected/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-kube-api-access-fblsn\") pod \"console-56f5b447fc-mjdnm\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.329949 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.329877 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:23.453751 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:23.453722 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f5b447fc-mjdnm"] Jun 03 16:48:23.456883 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:48:23.456851 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ea593e_ea59_4f18_b2e4_daca3b7ed4a2.slice/crio-9b7df26320bd9f1e4074db4ff0d406137071da7b400a7d23cbe81c22a301c990 WatchSource:0}: Error finding container 9b7df26320bd9f1e4074db4ff0d406137071da7b400a7d23cbe81c22a301c990: Status 404 returned error can't find the container with id 9b7df26320bd9f1e4074db4ff0d406137071da7b400a7d23cbe81c22a301c990 Jun 03 16:48:24.029471 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:24.029436 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f5b447fc-mjdnm" event={"ID":"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2","Type":"ContainerStarted","Data":"88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b"} Jun 03 16:48:24.029471 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:24.029472 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f5b447fc-mjdnm" event={"ID":"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2","Type":"ContainerStarted","Data":"9b7df26320bd9f1e4074db4ff0d406137071da7b400a7d23cbe81c22a301c990"} Jun 03 16:48:24.046226 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:24.046182 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-56f5b447fc-mjdnm" podStartSLOduration=2.046167424 podStartE2EDuration="2.046167424s" podCreationTimestamp="2026-06-03 16:48:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:48:24.045819626 +0000 UTC m=+276.429167046" watchObservedRunningTime="2026-06-03 16:48:24.046167424 +0000 UTC m=+276.429514848" Jun 03 16:48:28.631690 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:48:28.631650 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-tch6r" podUID="a904f235-adff-4af5-b66c-fee8a13025d5" Jun 03 16:48:28.631690 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:48:28.631664 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-x82rl" podUID="ba615b97-7ad1-40fe-868e-e6ec99612afe" Jun 03 16:48:29.042776 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:29.042709 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:48:29.042912 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:29.042708 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x82rl" Jun 03 16:48:32.567468 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.567432 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:48:32.567863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.567508 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:48:32.569967 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.569932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba615b97-7ad1-40fe-868e-e6ec99612afe-metrics-tls\") pod \"dns-default-x82rl\" (UID: \"ba615b97-7ad1-40fe-868e-e6ec99612afe\") " pod="openshift-dns/dns-default-x82rl" Jun 03 16:48:32.570084 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.569999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a904f235-adff-4af5-b66c-fee8a13025d5-cert\") pod \"ingress-canary-tch6r\" (UID: \"a904f235-adff-4af5-b66c-fee8a13025d5\") " pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:48:32.645625 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.645603 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6lvq\"" Jun 03 16:48:32.645923 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.645908 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vw2lw\"" Jun 03 16:48:32.654373 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.654353 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x82rl" Jun 03 16:48:32.654440 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.654375 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tch6r" Jun 03 16:48:32.785214 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.785187 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tch6r"] Jun 03 16:48:32.787788 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:48:32.787761 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda904f235_adff_4af5_b66c_fee8a13025d5.slice/crio-4c7466513672f67c9ce22fbe8c011989617c907cae514fb5976d21f52dd59ebd WatchSource:0}: Error finding container 4c7466513672f67c9ce22fbe8c011989617c907cae514fb5976d21f52dd59ebd: Status 404 returned error can't find the container with id 4c7466513672f67c9ce22fbe8c011989617c907cae514fb5976d21f52dd59ebd Jun 03 16:48:32.801427 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:32.801353 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x82rl"] Jun 03 16:48:32.803376 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:48:32.803351 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba615b97_7ad1_40fe_868e_e6ec99612afe.slice/crio-1963527bb915f7efa95a5c19c67db13165531d3a36f07bb81f819b5ba8858809 WatchSource:0}: Error finding container 1963527bb915f7efa95a5c19c67db13165531d3a36f07bb81f819b5ba8858809: Status 404 returned error can't find the container with id 1963527bb915f7efa95a5c19c67db13165531d3a36f07bb81f819b5ba8858809 Jun 03 16:48:33.054612 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:33.054573 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tch6r" event={"ID":"a904f235-adff-4af5-b66c-fee8a13025d5","Type":"ContainerStarted","Data":"4c7466513672f67c9ce22fbe8c011989617c907cae514fb5976d21f52dd59ebd"} Jun 03 16:48:33.055486 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:33.055461 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x82rl" event={"ID":"ba615b97-7ad1-40fe-868e-e6ec99612afe","Type":"ContainerStarted","Data":"1963527bb915f7efa95a5c19c67db13165531d3a36f07bb81f819b5ba8858809"} Jun 03 16:48:33.330259 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:33.330177 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:33.330259 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:33.330227 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:33.336269 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:33.336233 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:34.063981 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:34.063952 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:48:34.116391 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:34.116362 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645c8b888b-vws7l"] Jun 03 16:48:35.062528 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:35.062492 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x82rl" event={"ID":"ba615b97-7ad1-40fe-868e-e6ec99612afe","Type":"ContainerStarted","Data":"63c13520055e77d0c19845e8f247f0b189fbe101f4704ca2904e27c9a2eb83a6"} Jun 03 16:48:35.062528 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:35.062532 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x82rl" event={"ID":"ba615b97-7ad1-40fe-868e-e6ec99612afe","Type":"ContainerStarted","Data":"66e8a6b683adab2892988c899eb8a20653e814c8560b0ac681acadf142e82b53"} Jun 03 16:48:35.062786 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:35.062595 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x82rl" Jun 03 16:48:35.063974 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:35.063943 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tch6r" event={"ID":"a904f235-adff-4af5-b66c-fee8a13025d5","Type":"ContainerStarted","Data":"39d052574c649c189c22a918f913ee9770abd5c7c881968a46ad80b9065dc8b5"} Jun 03 16:48:35.087724 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:35.087674 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-x82rl" podStartSLOduration=251.339616294 podStartE2EDuration="4m13.08766311s" podCreationTimestamp="2026-06-03 16:44:22 +0000 UTC" firstStartedPulling="2026-06-03 16:48:32.805077711 +0000 UTC m=+285.188425109" lastFinishedPulling="2026-06-03 16:48:34.553124528 +0000 UTC m=+286.936471925" observedRunningTime="2026-06-03 16:48:35.085837242 +0000 UTC m=+287.469184660" watchObservedRunningTime="2026-06-03 16:48:35.08766311 +0000 UTC m=+287.471010529" Jun 03 16:48:35.101670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:35.101629 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tch6r" podStartSLOduration=251.334372357 podStartE2EDuration="4m13.101614129s" podCreationTimestamp="2026-06-03 16:44:22 +0000 UTC" firstStartedPulling="2026-06-03 16:48:32.789600465 +0000 UTC m=+285.172947862" lastFinishedPulling="2026-06-03 16:48:34.556842233 +0000 UTC m=+286.940189634" observedRunningTime="2026-06-03 16:48:35.100766416 +0000 UTC m=+287.484113836" watchObservedRunningTime="2026-06-03 16:48:35.101614129 +0000 UTC m=+287.484961551" Jun 03 16:48:45.069316 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:45.069282 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x82rl" Jun 03 16:48:47.999494 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:47.999463 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:48:48.000709 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:48.000505 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:48:48.009189 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:48.009173 2578 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 16:48:59.145665 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.145623 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-645c8b888b-vws7l" podUID="f6501044-9e22-4216-821f-bae364c66f99" containerName="console" containerID="cri-o://adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d" gracePeriod=15 Jun 03 16:48:59.377839 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.377818 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-645c8b888b-vws7l_f6501044-9e22-4216-821f-bae364c66f99/console/0.log" Jun 03 16:48:59.377958 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.377891 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:48:59.474141 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474072 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsqnh\" (UniqueName: \"kubernetes.io/projected/f6501044-9e22-4216-821f-bae364c66f99-kube-api-access-rsqnh\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474141 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474130 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-oauth-config\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474313 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474161 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-console-config\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474313 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474195 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-serving-cert\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474313 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474225 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-oauth-serving-cert\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474313 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474252 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-trusted-ca-bundle\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474313 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474288 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-service-ca\") pod \"f6501044-9e22-4216-821f-bae364c66f99\" (UID: \"f6501044-9e22-4216-821f-bae364c66f99\") " Jun 03 16:48:59.474706 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474680 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-console-config" (OuterVolumeSpecName: "console-config") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:48:59.474706 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474690 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:48:59.474841 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474728 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-service-ca" (OuterVolumeSpecName: "service-ca") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:48:59.474841 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.474782 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:48:59.476242 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.476218 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:48:59.476773 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.476754 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6501044-9e22-4216-821f-bae364c66f99-kube-api-access-rsqnh" (OuterVolumeSpecName: "kube-api-access-rsqnh") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "kube-api-access-rsqnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:48:59.476827 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.476771 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f6501044-9e22-4216-821f-bae364c66f99" (UID: "f6501044-9e22-4216-821f-bae364c66f99"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:48:59.575522 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575501 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsqnh\" (UniqueName: \"kubernetes.io/projected/f6501044-9e22-4216-821f-bae364c66f99-kube-api-access-rsqnh\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:48:59.575522 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575520 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-oauth-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:48:59.575670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575530 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-console-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:48:59.575670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575540 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6501044-9e22-4216-821f-bae364c66f99-console-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:48:59.575670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575550 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-oauth-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:48:59.575670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575581 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-trusted-ca-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:48:59.575670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:48:59.575595 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6501044-9e22-4216-821f-bae364c66f99-service-ca\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:49:00.137677 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.137649 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-645c8b888b-vws7l_f6501044-9e22-4216-821f-bae364c66f99/console/0.log" Jun 03 16:49:00.137851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.137695 2578 generic.go:358] "Generic (PLEG): container finished" podID="f6501044-9e22-4216-821f-bae364c66f99" containerID="adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d" exitCode=2 Jun 03 16:49:00.137851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.137727 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c8b888b-vws7l" event={"ID":"f6501044-9e22-4216-821f-bae364c66f99","Type":"ContainerDied","Data":"adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d"} Jun 03 16:49:00.137851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.137755 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c8b888b-vws7l" event={"ID":"f6501044-9e22-4216-821f-bae364c66f99","Type":"ContainerDied","Data":"7b81c70f5babff9e86d63e373901605a5d9c97452edad73d118c871ea0e0b4fe"} Jun 03 16:49:00.137851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.137759 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c8b888b-vws7l" Jun 03 16:49:00.137851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.137773 2578 scope.go:117] "RemoveContainer" containerID="adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d" Jun 03 16:49:00.145800 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.145783 2578 scope.go:117] "RemoveContainer" containerID="adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d" Jun 03 16:49:00.146151 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:49:00.146036 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d\": container with ID starting with adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d not found: ID does not exist" containerID="adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d" Jun 03 16:49:00.146151 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.146065 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d"} err="failed to get container status \"adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d\": rpc error: code = NotFound desc = could not find container \"adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d\": container with ID starting with adb0c5ff1e306b696bd776c914d7eb2e670cb538caa3f9012ca5ce4958f50a2d not found: ID does not exist" Jun 03 16:49:00.153488 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.153468 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645c8b888b-vws7l"] Jun 03 16:49:00.156968 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:00.156946 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-645c8b888b-vws7l"] Jun 03 16:49:02.098935 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:49:02.098892 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6501044-9e22-4216-821f-bae364c66f99" path="/var/lib/kubelet/pods/f6501044-9e22-4216-821f-bae364c66f99/volumes" Jun 03 16:52:02.440393 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.440356 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77"] Jun 03 16:52:02.440890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.440669 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6501044-9e22-4216-821f-bae364c66f99" containerName="console" Jun 03 16:52:02.440890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.440681 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6501044-9e22-4216-821f-bae364c66f99" containerName="console" Jun 03 16:52:02.440890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.440726 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6501044-9e22-4216-821f-bae364c66f99" containerName="console" Jun 03 16:52:02.443751 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.443735 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.445807 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.445785 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:52:02.445912 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.445788 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2wd4b\"" Jun 03 16:52:02.446190 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.446176 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:52:02.451272 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.451252 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77"] Jun 03 16:52:02.595501 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.595473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.595642 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.595541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.595642 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.595594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v74pj\" (UniqueName: \"kubernetes.io/projected/3b1e4a1c-d9f4-4025-bbee-77990916f33e-kube-api-access-v74pj\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.696533 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.696461 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.696533 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.696528 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.696704 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.696550 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v74pj\" (UniqueName: \"kubernetes.io/projected/3b1e4a1c-d9f4-4025-bbee-77990916f33e-kube-api-access-v74pj\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.696835 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.696816 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.696877 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.696847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.704116 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.704087 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v74pj\" (UniqueName: \"kubernetes.io/projected/3b1e4a1c-d9f4-4025-bbee-77990916f33e-kube-api-access-v74pj\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.754186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.754163 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:02.875313 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.875289 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77"] Jun 03 16:52:02.877884 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:02.877857 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1e4a1c_d9f4_4025_bbee_77990916f33e.slice/crio-4e6f5b3f8fc4be8f3eed11fa3867bfd082d29992af755adf398b23de96a5060e WatchSource:0}: Error finding container 4e6f5b3f8fc4be8f3eed11fa3867bfd082d29992af755adf398b23de96a5060e: Status 404 returned error can't find the container with id 4e6f5b3f8fc4be8f3eed11fa3867bfd082d29992af755adf398b23de96a5060e Jun 03 16:52:02.879660 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:02.879646 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:52:03.641188 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:03.641145 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" event={"ID":"3b1e4a1c-d9f4-4025-bbee-77990916f33e","Type":"ContainerStarted","Data":"4e6f5b3f8fc4be8f3eed11fa3867bfd082d29992af755adf398b23de96a5060e"} Jun 03 16:52:11.665689 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:11.665658 2578 generic.go:358] "Generic (PLEG): container finished" podID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerID="35a2714206d73f9a6de3deecb77dbff46285cae9a2fb47642be49cdf4ecd4e96" exitCode=0 Jun 03 16:52:11.666038 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:11.665748 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" event={"ID":"3b1e4a1c-d9f4-4025-bbee-77990916f33e","Type":"ContainerDied","Data":"35a2714206d73f9a6de3deecb77dbff46285cae9a2fb47642be49cdf4ecd4e96"} Jun 03 16:52:14.676525 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:14.676490 2578 generic.go:358] "Generic (PLEG): container finished" podID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerID="e9cc79d8488f7db9d9d1b7b9d83fc190e47f3765a8f01399030593ced36f1bc8" exitCode=0 Jun 03 16:52:14.676888 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:14.676587 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" event={"ID":"3b1e4a1c-d9f4-4025-bbee-77990916f33e","Type":"ContainerDied","Data":"e9cc79d8488f7db9d9d1b7b9d83fc190e47f3765a8f01399030593ced36f1bc8"} Jun 03 16:52:20.375580 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.375527 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-8687b5b458-kcxfj"] Jun 03 16:52:20.381924 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.381898 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.394714 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.394692 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8687b5b458-kcxfj"] Jun 03 16:52:20.444289 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5de392ed-47a2-4afb-ba88-3147c2469b08-console-serving-cert\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.444421 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-console-config\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.444480 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-service-ca\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.444532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5de392ed-47a2-4afb-ba88-3147c2469b08-console-oauth-config\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.444590 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-trusted-ca-bundle\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.444646 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-oauth-serving-cert\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.444699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.444661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6hjb\" (UniqueName: \"kubernetes.io/projected/5de392ed-47a2-4afb-ba88-3147c2469b08-kube-api-access-l6hjb\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545381 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545352 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5de392ed-47a2-4afb-ba88-3147c2469b08-console-oauth-config\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545381 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545391 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-trusted-ca-bundle\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545432 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-oauth-serving-cert\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545456 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6hjb\" (UniqueName: \"kubernetes.io/projected/5de392ed-47a2-4afb-ba88-3147c2469b08-kube-api-access-l6hjb\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545490 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5de392ed-47a2-4afb-ba88-3147c2469b08-console-serving-cert\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545514 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-console-config\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.545583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.545542 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-service-ca\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.546298 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.546269 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-service-ca\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.546425 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.546366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-console-config\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.546622 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.546596 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-oauth-serving-cert\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.546728 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.546616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5de392ed-47a2-4afb-ba88-3147c2469b08-trusted-ca-bundle\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.548286 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.548264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5de392ed-47a2-4afb-ba88-3147c2469b08-console-serving-cert\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.548397 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.548324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5de392ed-47a2-4afb-ba88-3147c2469b08-console-oauth-config\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.552863 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.552841 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6hjb\" (UniqueName: \"kubernetes.io/projected/5de392ed-47a2-4afb-ba88-3147c2469b08-kube-api-access-l6hjb\") pod \"console-8687b5b458-kcxfj\" (UID: \"5de392ed-47a2-4afb-ba88-3147c2469b08\") " pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.691989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.691900 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:20.933727 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:20.933701 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8687b5b458-kcxfj"] Jun 03 16:52:20.936768 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:20.936745 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de392ed_47a2_4afb_ba88_3147c2469b08.slice/crio-96fd72004c604f67f5842c7bc0b9e07de614e0562aef685f1535cf5fd82df52f WatchSource:0}: Error finding container 96fd72004c604f67f5842c7bc0b9e07de614e0562aef685f1535cf5fd82df52f: Status 404 returned error can't find the container with id 96fd72004c604f67f5842c7bc0b9e07de614e0562aef685f1535cf5fd82df52f Jun 03 16:52:21.699547 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:21.699514 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8687b5b458-kcxfj" event={"ID":"5de392ed-47a2-4afb-ba88-3147c2469b08","Type":"ContainerStarted","Data":"811a84db14f7934255b133eb08d8e77c2fb1a59ee7c15e5d7aab0fa106b5bbc9"} Jun 03 16:52:21.699933 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:21.699574 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8687b5b458-kcxfj" event={"ID":"5de392ed-47a2-4afb-ba88-3147c2469b08","Type":"ContainerStarted","Data":"96fd72004c604f67f5842c7bc0b9e07de614e0562aef685f1535cf5fd82df52f"} Jun 03 16:52:21.701327 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:21.701300 2578 generic.go:358] "Generic (PLEG): container finished" podID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerID="55c86783ffc8a38171556f5448618ea02085200fa8be5eaec34e1b7ff430e623" exitCode=0 Jun 03 16:52:21.701437 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:21.701371 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" event={"ID":"3b1e4a1c-d9f4-4025-bbee-77990916f33e","Type":"ContainerDied","Data":"55c86783ffc8a38171556f5448618ea02085200fa8be5eaec34e1b7ff430e623"} Jun 03 16:52:21.715543 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:21.715499 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-8687b5b458-kcxfj" podStartSLOduration=1.71548621 podStartE2EDuration="1.71548621s" podCreationTimestamp="2026-06-03 16:52:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:52:21.713663326 +0000 UTC m=+514.097010745" watchObservedRunningTime="2026-06-03 16:52:21.71548621 +0000 UTC m=+514.098833629" Jun 03 16:52:22.821962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.821938 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:22.858791 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.858764 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-bundle\") pod \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " Jun 03 16:52:22.858926 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.858816 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-util\") pod \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " Jun 03 16:52:22.858926 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.858839 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v74pj\" (UniqueName: \"kubernetes.io/projected/3b1e4a1c-d9f4-4025-bbee-77990916f33e-kube-api-access-v74pj\") pod \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\" (UID: \"3b1e4a1c-d9f4-4025-bbee-77990916f33e\") " Jun 03 16:52:22.859367 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.859343 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-bundle" (OuterVolumeSpecName: "bundle") pod "3b1e4a1c-d9f4-4025-bbee-77990916f33e" (UID: "3b1e4a1c-d9f4-4025-bbee-77990916f33e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:52:22.860908 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.860885 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b1e4a1c-d9f4-4025-bbee-77990916f33e-kube-api-access-v74pj" (OuterVolumeSpecName: "kube-api-access-v74pj") pod "3b1e4a1c-d9f4-4025-bbee-77990916f33e" (UID: "3b1e4a1c-d9f4-4025-bbee-77990916f33e"). InnerVolumeSpecName "kube-api-access-v74pj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:52:22.865814 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.865790 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-util" (OuterVolumeSpecName: "util") pod "3b1e4a1c-d9f4-4025-bbee-77990916f33e" (UID: "3b1e4a1c-d9f4-4025-bbee-77990916f33e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:52:22.959396 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.959336 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:22.959396 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.959360 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b1e4a1c-d9f4-4025-bbee-77990916f33e-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:22.959396 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:22.959370 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v74pj\" (UniqueName: \"kubernetes.io/projected/3b1e4a1c-d9f4-4025-bbee-77990916f33e-kube-api-access-v74pj\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:23.709015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:23.708982 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" event={"ID":"3b1e4a1c-d9f4-4025-bbee-77990916f33e","Type":"ContainerDied","Data":"4e6f5b3f8fc4be8f3eed11fa3867bfd082d29992af755adf398b23de96a5060e"} Jun 03 16:52:23.709015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:23.709006 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzjt77" Jun 03 16:52:23.709015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:23.709018 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e6f5b3f8fc4be8f3eed11fa3867bfd082d29992af755adf398b23de96a5060e" Jun 03 16:52:30.187428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187391 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm"] Jun 03 16:52:30.187825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187746 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="extract" Jun 03 16:52:30.187825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187761 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="extract" Jun 03 16:52:30.187825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187772 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="util" Jun 03 16:52:30.187825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187778 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="util" Jun 03 16:52:30.187825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187784 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="pull" Jun 03 16:52:30.187825 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187789 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="pull" Jun 03 16:52:30.188013 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.187840 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="3b1e4a1c-d9f4-4025-bbee-77990916f33e" containerName="extract" Jun 03 16:52:30.193788 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.193770 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.195782 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.195756 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:52:30.195876 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.195826 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 03 16:52:30.195938 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.195921 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g8fz4\"" Jun 03 16:52:30.202922 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.202901 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm"] Jun 03 16:52:30.203524 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.203501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dd796b5-233b-4755-9950-83c64f85dbe5-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-nmqvm\" (UID: \"6dd796b5-233b-4755-9950-83c64f85dbe5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.203640 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.203583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8j87\" (UniqueName: \"kubernetes.io/projected/6dd796b5-233b-4755-9950-83c64f85dbe5-kube-api-access-r8j87\") pod \"cert-manager-operator-controller-manager-54b9655956-nmqvm\" (UID: \"6dd796b5-233b-4755-9950-83c64f85dbe5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.304517 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.304483 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8j87\" (UniqueName: \"kubernetes.io/projected/6dd796b5-233b-4755-9950-83c64f85dbe5-kube-api-access-r8j87\") pod \"cert-manager-operator-controller-manager-54b9655956-nmqvm\" (UID: \"6dd796b5-233b-4755-9950-83c64f85dbe5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.304705 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.304594 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dd796b5-233b-4755-9950-83c64f85dbe5-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-nmqvm\" (UID: \"6dd796b5-233b-4755-9950-83c64f85dbe5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.304929 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.304913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dd796b5-233b-4755-9950-83c64f85dbe5-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-nmqvm\" (UID: \"6dd796b5-233b-4755-9950-83c64f85dbe5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.311859 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.311830 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8j87\" (UniqueName: \"kubernetes.io/projected/6dd796b5-233b-4755-9950-83c64f85dbe5-kube-api-access-r8j87\") pod \"cert-manager-operator-controller-manager-54b9655956-nmqvm\" (UID: \"6dd796b5-233b-4755-9950-83c64f85dbe5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.502821 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.502734 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" Jun 03 16:52:30.629478 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.629444 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm"] Jun 03 16:52:30.630681 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:30.630657 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd796b5_233b_4755_9950_83c64f85dbe5.slice/crio-d6327e9a0e742c3a570c4f18bd79b0cffd022d92c1a985495ac1ac300da6ba69 WatchSource:0}: Error finding container d6327e9a0e742c3a570c4f18bd79b0cffd022d92c1a985495ac1ac300da6ba69: Status 404 returned error can't find the container with id d6327e9a0e742c3a570c4f18bd79b0cffd022d92c1a985495ac1ac300da6ba69 Jun 03 16:52:30.692957 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.692933 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:30.693056 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.692966 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:30.697341 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.697323 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:30.730577 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.730531 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" event={"ID":"6dd796b5-233b-4755-9950-83c64f85dbe5","Type":"ContainerStarted","Data":"d6327e9a0e742c3a570c4f18bd79b0cffd022d92c1a985495ac1ac300da6ba69"} Jun 03 16:52:30.734537 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.734518 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8687b5b458-kcxfj" Jun 03 16:52:30.782328 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:30.782254 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f5b447fc-mjdnm"] Jun 03 16:52:32.741074 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:32.741041 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" event={"ID":"6dd796b5-233b-4755-9950-83c64f85dbe5","Type":"ContainerStarted","Data":"28bc713138eacae287e91a556ba84088f763d08092e24d71a9aeb8e224aebd60"} Jun 03 16:52:32.764735 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:32.763234 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nmqvm" podStartSLOduration=0.752597788 podStartE2EDuration="2.763208097s" podCreationTimestamp="2026-06-03 16:52:30 +0000 UTC" firstStartedPulling="2026-06-03 16:52:30.632961185 +0000 UTC m=+523.016308582" lastFinishedPulling="2026-06-03 16:52:32.643571485 +0000 UTC m=+525.026918891" observedRunningTime="2026-06-03 16:52:32.757640084 +0000 UTC m=+525.140987504" watchObservedRunningTime="2026-06-03 16:52:32.763208097 +0000 UTC m=+525.146555522" Jun 03 16:52:34.076804 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.076766 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj"] Jun 03 16:52:34.080308 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.080287 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.082381 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.082355 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:52:34.082754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.082738 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:52:34.082846 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.082750 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2wd4b\"" Jun 03 16:52:34.089102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.089078 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj"] Jun 03 16:52:34.134961 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.134935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwpcx\" (UniqueName: \"kubernetes.io/projected/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-kube-api-access-kwpcx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.135066 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.134971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.135066 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.135002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.236202 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.236177 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kwpcx\" (UniqueName: \"kubernetes.io/projected/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-kube-api-access-kwpcx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.236290 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.236208 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.236290 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.236237 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.236643 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.236624 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.236688 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.236638 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.244170 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.244146 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwpcx\" (UniqueName: \"kubernetes.io/projected/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-kube-api-access-kwpcx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.392514 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.392489 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:34.512962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.512923 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj"] Jun 03 16:52:34.515714 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:34.515685 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0b794f_1432_4bdf_8a1e_83ccd4364cf4.slice/crio-655e08c4381fc4b044297bdb190597dd8ce38d631411e0564733dec257fa7942 WatchSource:0}: Error finding container 655e08c4381fc4b044297bdb190597dd8ce38d631411e0564733dec257fa7942: Status 404 returned error can't find the container with id 655e08c4381fc4b044297bdb190597dd8ce38d631411e0564733dec257fa7942 Jun 03 16:52:34.633019 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:52:34.632993 2578 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0b794f_1432_4bdf_8a1e_83ccd4364cf4.slice/crio-conmon-7bae1299485c0cd1a328c72cd19ba1d578a0cf4e8be9b875b47ed5c406a6c6af.scope\": RecentStats: unable to find data in memory cache]" Jun 03 16:52:34.748477 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.748414 2578 generic.go:358] "Generic (PLEG): container finished" podID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerID="7bae1299485c0cd1a328c72cd19ba1d578a0cf4e8be9b875b47ed5c406a6c6af" exitCode=0 Jun 03 16:52:34.748615 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.748508 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" event={"ID":"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4","Type":"ContainerDied","Data":"7bae1299485c0cd1a328c72cd19ba1d578a0cf4e8be9b875b47ed5c406a6c6af"} Jun 03 16:52:34.748615 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:34.748550 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" event={"ID":"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4","Type":"ContainerStarted","Data":"655e08c4381fc4b044297bdb190597dd8ce38d631411e0564733dec257fa7942"} Jun 03 16:52:37.762758 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:37.762723 2578 generic.go:358] "Generic (PLEG): container finished" podID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerID="436892e8a77fe0308bcf2afd611f412964bf17d6bf0ee294303706a2c6a75a3f" exitCode=0 Jun 03 16:52:37.763137 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:37.762809 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" event={"ID":"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4","Type":"ContainerDied","Data":"436892e8a77fe0308bcf2afd611f412964bf17d6bf0ee294303706a2c6a75a3f"} Jun 03 16:52:38.768652 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.768611 2578 generic.go:358] "Generic (PLEG): container finished" podID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerID="f4c8e1e622b8f4dbdfe3208a7222cfc0539d6807fad3c77d2a0976a6650912ed" exitCode=0 Jun 03 16:52:38.769087 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.768697 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" event={"ID":"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4","Type":"ContainerDied","Data":"f4c8e1e622b8f4dbdfe3208a7222cfc0539d6807fad3c77d2a0976a6650912ed"} Jun 03 16:52:38.777084 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.777057 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-mf7qt"] Jun 03 16:52:38.780311 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.780295 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:38.782144 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.782115 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 03 16:52:38.782245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.782173 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 03 16:52:38.782245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.782214 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xm8t2\"" Jun 03 16:52:38.789610 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.789587 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-mf7qt"] Jun 03 16:52:38.872672 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.872648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/429baee0-42f3-4b89-b385-2310e34ba1f6-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-mf7qt\" (UID: \"429baee0-42f3-4b89-b385-2310e34ba1f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:38.872777 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.872726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n4dm\" (UniqueName: \"kubernetes.io/projected/429baee0-42f3-4b89-b385-2310e34ba1f6-kube-api-access-9n4dm\") pod \"cert-manager-cainjector-68b757865b-mf7qt\" (UID: \"429baee0-42f3-4b89-b385-2310e34ba1f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:38.973561 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.973540 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9n4dm\" (UniqueName: \"kubernetes.io/projected/429baee0-42f3-4b89-b385-2310e34ba1f6-kube-api-access-9n4dm\") pod \"cert-manager-cainjector-68b757865b-mf7qt\" (UID: \"429baee0-42f3-4b89-b385-2310e34ba1f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:38.973663 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.973592 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/429baee0-42f3-4b89-b385-2310e34ba1f6-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-mf7qt\" (UID: \"429baee0-42f3-4b89-b385-2310e34ba1f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:38.980475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.980456 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/429baee0-42f3-4b89-b385-2310e34ba1f6-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-mf7qt\" (UID: \"429baee0-42f3-4b89-b385-2310e34ba1f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:38.980734 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:38.980713 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n4dm\" (UniqueName: \"kubernetes.io/projected/429baee0-42f3-4b89-b385-2310e34ba1f6-kube-api-access-9n4dm\") pod \"cert-manager-cainjector-68b757865b-mf7qt\" (UID: \"429baee0-42f3-4b89-b385-2310e34ba1f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:39.105545 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.105523 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" Jun 03 16:52:39.233351 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.233320 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-mf7qt"] Jun 03 16:52:39.235694 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:39.235663 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod429baee0_42f3_4b89_b385_2310e34ba1f6.slice/crio-c4ca3f42387823632a4dc004bdc0d752401bd69dd89df95ce3224a046d1e479e WatchSource:0}: Error finding container c4ca3f42387823632a4dc004bdc0d752401bd69dd89df95ce3224a046d1e479e: Status 404 returned error can't find the container with id c4ca3f42387823632a4dc004bdc0d752401bd69dd89df95ce3224a046d1e479e Jun 03 16:52:39.773072 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.773038 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" event={"ID":"429baee0-42f3-4b89-b385-2310e34ba1f6","Type":"ContainerStarted","Data":"c4ca3f42387823632a4dc004bdc0d752401bd69dd89df95ce3224a046d1e479e"} Jun 03 16:52:39.901437 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.901413 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:39.984254 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.984227 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-util\") pod \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " Jun 03 16:52:39.984405 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.984298 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-bundle\") pod \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " Jun 03 16:52:39.984405 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.984363 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwpcx\" (UniqueName: \"kubernetes.io/projected/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-kube-api-access-kwpcx\") pod \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\" (UID: \"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4\") " Jun 03 16:52:39.984808 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.984782 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-bundle" (OuterVolumeSpecName: "bundle") pod "aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" (UID: "aa0b794f-1432-4bdf-8a1e-83ccd4364cf4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:52:39.986997 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:39.986842 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-kube-api-access-kwpcx" (OuterVolumeSpecName: "kube-api-access-kwpcx") pod "aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" (UID: "aa0b794f-1432-4bdf-8a1e-83ccd4364cf4"). InnerVolumeSpecName "kube-api-access-kwpcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:52:40.066843 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.066782 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-util" (OuterVolumeSpecName: "util") pod "aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" (UID: "aa0b794f-1432-4bdf-8a1e-83ccd4364cf4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:52:40.085824 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.085797 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:40.085824 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.085825 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:40.085983 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.085836 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwpcx\" (UniqueName: \"kubernetes.io/projected/aa0b794f-1432-4bdf-8a1e-83ccd4364cf4-kube-api-access-kwpcx\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:40.778754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.778717 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" event={"ID":"aa0b794f-1432-4bdf-8a1e-83ccd4364cf4","Type":"ContainerDied","Data":"655e08c4381fc4b044297bdb190597dd8ce38d631411e0564733dec257fa7942"} Jun 03 16:52:40.778754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.778757 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="655e08c4381fc4b044297bdb190597dd8ce38d631411e0564733dec257fa7942" Jun 03 16:52:40.779127 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:40.778794 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmghj" Jun 03 16:52:41.617578 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617518 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-kdgfb"] Jun 03 16:52:41.617847 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617834 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="util" Jun 03 16:52:41.617897 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617849 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="util" Jun 03 16:52:41.617897 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617863 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="extract" Jun 03 16:52:41.617897 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617868 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="extract" Jun 03 16:52:41.617897 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617880 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="pull" Jun 03 16:52:41.617897 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617886 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="pull" Jun 03 16:52:41.618042 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.617935 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa0b794f-1432-4bdf-8a1e-83ccd4364cf4" containerName="extract" Jun 03 16:52:41.620638 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.620619 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.622462 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.622446 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5j778\"" Jun 03 16:52:41.626390 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.626352 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-kdgfb"] Jun 03 16:52:41.701576 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.701540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae0ba52c-111e-4628-9db1-cf62a4cb479a-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-kdgfb\" (UID: \"ae0ba52c-111e-4628-9db1-cf62a4cb479a\") " pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.701667 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.701594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbzzg\" (UniqueName: \"kubernetes.io/projected/ae0ba52c-111e-4628-9db1-cf62a4cb479a-kube-api-access-bbzzg\") pod \"cert-manager-webhook-587ccfb98-kdgfb\" (UID: \"ae0ba52c-111e-4628-9db1-cf62a4cb479a\") " pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.785686 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.785658 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" event={"ID":"429baee0-42f3-4b89-b385-2310e34ba1f6","Type":"ContainerStarted","Data":"2615fb6946dd3b0398cef1bafc8cc1211ebd1dbe81f3a7784464566680655385"} Jun 03 16:52:41.799718 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.799678 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-mf7qt" podStartSLOduration=1.394104658 podStartE2EDuration="3.799665203s" podCreationTimestamp="2026-06-03 16:52:38 +0000 UTC" firstStartedPulling="2026-06-03 16:52:39.237594932 +0000 UTC m=+531.620942329" lastFinishedPulling="2026-06-03 16:52:41.643155474 +0000 UTC m=+534.026502874" observedRunningTime="2026-06-03 16:52:41.79839055 +0000 UTC m=+534.181737969" watchObservedRunningTime="2026-06-03 16:52:41.799665203 +0000 UTC m=+534.183012622" Jun 03 16:52:41.802576 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.802532 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae0ba52c-111e-4628-9db1-cf62a4cb479a-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-kdgfb\" (UID: \"ae0ba52c-111e-4628-9db1-cf62a4cb479a\") " pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.802683 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.802588 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbzzg\" (UniqueName: \"kubernetes.io/projected/ae0ba52c-111e-4628-9db1-cf62a4cb479a-kube-api-access-bbzzg\") pod \"cert-manager-webhook-587ccfb98-kdgfb\" (UID: \"ae0ba52c-111e-4628-9db1-cf62a4cb479a\") " pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.810371 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.810347 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae0ba52c-111e-4628-9db1-cf62a4cb479a-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-kdgfb\" (UID: \"ae0ba52c-111e-4628-9db1-cf62a4cb479a\") " pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.810717 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.810679 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbzzg\" (UniqueName: \"kubernetes.io/projected/ae0ba52c-111e-4628-9db1-cf62a4cb479a-kube-api-access-bbzzg\") pod \"cert-manager-webhook-587ccfb98-kdgfb\" (UID: \"ae0ba52c-111e-4628-9db1-cf62a4cb479a\") " pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:41.930388 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:41.930297 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:42.084578 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:42.082681 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-kdgfb"] Jun 03 16:52:42.088825 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:42.088796 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae0ba52c_111e_4628_9db1_cf62a4cb479a.slice/crio-bab5b861b2b79092a778d5076327226a91978fbe4ec3f7c48afb1bc97a025537 WatchSource:0}: Error finding container bab5b861b2b79092a778d5076327226a91978fbe4ec3f7c48afb1bc97a025537: Status 404 returned error can't find the container with id bab5b861b2b79092a778d5076327226a91978fbe4ec3f7c48afb1bc97a025537 Jun 03 16:52:42.790876 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:42.790833 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" event={"ID":"ae0ba52c-111e-4628-9db1-cf62a4cb479a","Type":"ContainerStarted","Data":"fd5c1f22894860ff416cda40adc407faed4d2b4ebc51d022a6b03cf942bcbc3b"} Jun 03 16:52:42.790876 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:42.790875 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" event={"ID":"ae0ba52c-111e-4628-9db1-cf62a4cb479a","Type":"ContainerStarted","Data":"bab5b861b2b79092a778d5076327226a91978fbe4ec3f7c48afb1bc97a025537"} Jun 03 16:52:42.791360 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:42.791023 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:42.804753 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:42.804693 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" podStartSLOduration=1.8046784009999999 podStartE2EDuration="1.804678401s" podCreationTimestamp="2026-06-03 16:52:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:52:42.803817134 +0000 UTC m=+535.187164552" watchObservedRunningTime="2026-06-03 16:52:42.804678401 +0000 UTC m=+535.188025820" Jun 03 16:52:48.796120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:48.796087 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-kdgfb" Jun 03 16:52:54.943019 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:54.942967 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77"] Jun 03 16:52:54.950234 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:54.950208 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:54.952687 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:54.952664 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:52:54.952917 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:54.952898 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77"] Jun 03 16:52:54.953111 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:54.953091 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2wd4b\"" Jun 03 16:52:54.953111 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:54.953097 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:52:55.007477 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.007447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.007619 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.007481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.007619 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.007608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xrgl\" (UniqueName: \"kubernetes.io/projected/2b83f833-5c5a-4987-9137-bccce43d79e4-kube-api-access-7xrgl\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.109006 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.108975 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.109006 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.109009 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.109207 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.109064 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7xrgl\" (UniqueName: \"kubernetes.io/projected/2b83f833-5c5a-4987-9137-bccce43d79e4-kube-api-access-7xrgl\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.109418 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.109396 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.109418 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.109410 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.117099 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.117078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xrgl\" (UniqueName: \"kubernetes.io/projected/2b83f833-5c5a-4987-9137-bccce43d79e4-kube-api-access-7xrgl\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.261152 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.261069 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:55.382797 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.382772 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77"] Jun 03 16:52:55.385133 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:52:55.385106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b83f833_5c5a_4987_9137_bccce43d79e4.slice/crio-f2112f9ace4591e79b50002b6cdeefe8f7c775595da52dbff8593f8a3b7dd0be WatchSource:0}: Error finding container f2112f9ace4591e79b50002b6cdeefe8f7c775595da52dbff8593f8a3b7dd0be: Status 404 returned error can't find the container with id f2112f9ace4591e79b50002b6cdeefe8f7c775595da52dbff8593f8a3b7dd0be Jun 03 16:52:55.802390 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.802356 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-56f5b447fc-mjdnm" podUID="45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" containerName="console" containerID="cri-o://88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b" gracePeriod=15 Jun 03 16:52:55.832778 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.832749 2578 generic.go:358] "Generic (PLEG): container finished" podID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerID="e9f9a863a7791e4eaf69f3152f1cd3d0bf2b037829dded138658d3b04e7665eb" exitCode=0 Jun 03 16:52:55.832893 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.832796 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" event={"ID":"2b83f833-5c5a-4987-9137-bccce43d79e4","Type":"ContainerDied","Data":"e9f9a863a7791e4eaf69f3152f1cd3d0bf2b037829dded138658d3b04e7665eb"} Jun 03 16:52:55.832893 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:55.832819 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" event={"ID":"2b83f833-5c5a-4987-9137-bccce43d79e4","Type":"ContainerStarted","Data":"f2112f9ace4591e79b50002b6cdeefe8f7c775595da52dbff8593f8a3b7dd0be"} Jun 03 16:52:56.028650 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.028630 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56f5b447fc-mjdnm_45ea593e-ea59-4f18-b2e4-daca3b7ed4a2/console/0.log" Jun 03 16:52:56.028941 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.028687 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:52:56.116158 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116129 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-config\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116325 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116169 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-serving-cert\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116325 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116192 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-service-ca\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116325 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116218 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-oauth-config\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116512 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116350 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-trusted-ca-bundle\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116512 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116438 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-oauth-serving-cert\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116512 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116472 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fblsn\" (UniqueName: \"kubernetes.io/projected/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-kube-api-access-fblsn\") pod \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\" (UID: \"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2\") " Jun 03 16:52:56.116691 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116512 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-config" (OuterVolumeSpecName: "console-config") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:52:56.116691 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116638 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-service-ca" (OuterVolumeSpecName: "service-ca") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:52:56.116823 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116733 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:52:56.116823 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116794 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:52:56.116905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116862 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-oauth-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.116905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116878 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.116905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116893 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-service-ca\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.116905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.116905 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-trusted-ca-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.118394 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.118370 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:52:56.118659 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.118639 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-kube-api-access-fblsn" (OuterVolumeSpecName: "kube-api-access-fblsn") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "kube-api-access-fblsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:52:56.118718 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.118695 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" (UID: "45ea593e-ea59-4f18-b2e4-daca3b7ed4a2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:52:56.218123 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.218089 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-serving-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.218123 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.218116 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-console-oauth-config\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.218123 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.218126 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fblsn\" (UniqueName: \"kubernetes.io/projected/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2-kube-api-access-fblsn\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:56.837831 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.837805 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56f5b447fc-mjdnm_45ea593e-ea59-4f18-b2e4-daca3b7ed4a2/console/0.log" Jun 03 16:52:56.838010 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.837848 2578 generic.go:358] "Generic (PLEG): container finished" podID="45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" containerID="88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b" exitCode=2 Jun 03 16:52:56.838010 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.837887 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f5b447fc-mjdnm" event={"ID":"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2","Type":"ContainerDied","Data":"88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b"} Jun 03 16:52:56.838010 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.837936 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f5b447fc-mjdnm" event={"ID":"45ea593e-ea59-4f18-b2e4-daca3b7ed4a2","Type":"ContainerDied","Data":"9b7df26320bd9f1e4074db4ff0d406137071da7b400a7d23cbe81c22a301c990"} Jun 03 16:52:56.838010 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.837950 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f5b447fc-mjdnm" Jun 03 16:52:56.838193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.837957 2578 scope.go:117] "RemoveContainer" containerID="88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b" Jun 03 16:52:56.839657 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.839621 2578 generic.go:358] "Generic (PLEG): container finished" podID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerID="098d0f3f5c77d457640513d14a3b0d3b8576486d28f50b5826c1f7b11c4bc110" exitCode=0 Jun 03 16:52:56.839753 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.839677 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" event={"ID":"2b83f833-5c5a-4987-9137-bccce43d79e4","Type":"ContainerDied","Data":"098d0f3f5c77d457640513d14a3b0d3b8576486d28f50b5826c1f7b11c4bc110"} Jun 03 16:52:56.846939 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.846916 2578 scope.go:117] "RemoveContainer" containerID="88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b" Jun 03 16:52:56.847200 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:52:56.847179 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b\": container with ID starting with 88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b not found: ID does not exist" containerID="88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b" Jun 03 16:52:56.847253 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.847209 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b"} err="failed to get container status \"88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b\": rpc error: code = NotFound desc = could not find container \"88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b\": container with ID starting with 88e7b88c46cc6dea8dc5a599dd23484725f9d5b0beac36fdfe480100db10990b not found: ID does not exist" Jun 03 16:52:56.872021 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.871995 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f5b447fc-mjdnm"] Jun 03 16:52:56.875942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:56.875914 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f5b447fc-mjdnm"] Jun 03 16:52:57.845765 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:57.845731 2578 generic.go:358] "Generic (PLEG): container finished" podID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerID="c0242cddaf7af4aeebfca1099e6db76e4c4cabb4d568db98f83d52e325e31ca7" exitCode=0 Jun 03 16:52:57.846177 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:57.845825 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" event={"ID":"2b83f833-5c5a-4987-9137-bccce43d79e4","Type":"ContainerDied","Data":"c0242cddaf7af4aeebfca1099e6db76e4c4cabb4d568db98f83d52e325e31ca7"} Jun 03 16:52:58.100174 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:58.100097 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" path="/var/lib/kubelet/pods/45ea593e-ea59-4f18-b2e4-daca3b7ed4a2/volumes" Jun 03 16:52:58.970748 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:58.970726 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:59.042217 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.042186 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-bundle\") pod \"2b83f833-5c5a-4987-9137-bccce43d79e4\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " Jun 03 16:52:59.042379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.042244 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xrgl\" (UniqueName: \"kubernetes.io/projected/2b83f833-5c5a-4987-9137-bccce43d79e4-kube-api-access-7xrgl\") pod \"2b83f833-5c5a-4987-9137-bccce43d79e4\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " Jun 03 16:52:59.042379 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.042300 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-util\") pod \"2b83f833-5c5a-4987-9137-bccce43d79e4\" (UID: \"2b83f833-5c5a-4987-9137-bccce43d79e4\") " Jun 03 16:52:59.042947 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.042918 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-bundle" (OuterVolumeSpecName: "bundle") pod "2b83f833-5c5a-4987-9137-bccce43d79e4" (UID: "2b83f833-5c5a-4987-9137-bccce43d79e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:52:59.044306 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.044282 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b83f833-5c5a-4987-9137-bccce43d79e4-kube-api-access-7xrgl" (OuterVolumeSpecName: "kube-api-access-7xrgl") pod "2b83f833-5c5a-4987-9137-bccce43d79e4" (UID: "2b83f833-5c5a-4987-9137-bccce43d79e4"). InnerVolumeSpecName "kube-api-access-7xrgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:52:59.048101 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.048075 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-util" (OuterVolumeSpecName: "util") pod "2b83f833-5c5a-4987-9137-bccce43d79e4" (UID: "2b83f833-5c5a-4987-9137-bccce43d79e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:52:59.142937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.142859 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:59.142937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.142896 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xrgl\" (UniqueName: \"kubernetes.io/projected/2b83f833-5c5a-4987-9137-bccce43d79e4-kube-api-access-7xrgl\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:59.142937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.142908 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b83f833-5c5a-4987-9137-bccce43d79e4-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:52:59.855406 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.855379 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" Jun 03 16:52:59.855583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.855382 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjln77" event={"ID":"2b83f833-5c5a-4987-9137-bccce43d79e4","Type":"ContainerDied","Data":"f2112f9ace4591e79b50002b6cdeefe8f7c775595da52dbff8593f8a3b7dd0be"} Jun 03 16:52:59.855583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:52:59.855486 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2112f9ace4591e79b50002b6cdeefe8f7c775595da52dbff8593f8a3b7dd0be" Jun 03 16:53:06.547774 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.547737 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6"] Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548168 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="util" Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548186 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="util" Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548198 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="extract" Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548206 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="extract" Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548238 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" containerName="console" Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548246 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" containerName="console" Jun 03 16:53:06.548252 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548254 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="pull" Jun 03 16:53:06.548634 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548262 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="pull" Jun 03 16:53:06.548634 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548358 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="45ea593e-ea59-4f18-b2e4-daca3b7ed4a2" containerName="console" Jun 03 16:53:06.548634 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.548371 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b83f833-5c5a-4987-9137-bccce43d79e4" containerName="extract" Jun 03 16:53:06.551368 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.551346 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.553841 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.553820 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 16:53:06.554101 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.554076 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cmn8q\"" Jun 03 16:53:06.554356 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.554342 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 16:53:06.554424 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.554381 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 16:53:06.554480 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.554458 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 16:53:06.586503 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.586472 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6"] Jun 03 16:53:06.707686 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.707642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eccef2f7-2352-415f-8eda-cee467d6b719-tmp\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.707882 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.707705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccef2f7-2352-415f-8eda-cee467d6b719-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.707882 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.707783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eccef2f7-2352-415f-8eda-cee467d6b719-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.707882 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.707867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn5g8\" (UniqueName: \"kubernetes.io/projected/eccef2f7-2352-415f-8eda-cee467d6b719-kube-api-access-dn5g8\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.708036 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.707921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eccef2f7-2352-415f-8eda-cee467d6b719-manifests\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.808670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.808596 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eccef2f7-2352-415f-8eda-cee467d6b719-manifests\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.808670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.808648 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eccef2f7-2352-415f-8eda-cee467d6b719-tmp\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.808670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.808665 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccef2f7-2352-415f-8eda-cee467d6b719-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.808858 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.808689 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eccef2f7-2352-415f-8eda-cee467d6b719-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.808858 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.808737 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dn5g8\" (UniqueName: \"kubernetes.io/projected/eccef2f7-2352-415f-8eda-cee467d6b719-kube-api-access-dn5g8\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.809072 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.809046 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eccef2f7-2352-415f-8eda-cee467d6b719-manifests\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.810981 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.810950 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eccef2f7-2352-415f-8eda-cee467d6b719-tmp\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.811181 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.811165 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccef2f7-2352-415f-8eda-cee467d6b719-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.811221 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.811201 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eccef2f7-2352-415f-8eda-cee467d6b719-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.819409 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.819389 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn5g8\" (UniqueName: \"kubernetes.io/projected/eccef2f7-2352-415f-8eda-cee467d6b719-kube-api-access-dn5g8\") pod \"opendatahub-operator-controller-manager-d99597487-4jph6\" (UID: \"eccef2f7-2352-415f-8eda-cee467d6b719\") " pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:06.861385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:06.861355 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:07.000868 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.000834 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6"] Jun 03 16:53:07.005149 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:53:07.005115 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccef2f7_2352_415f_8eda_cee467d6b719.slice/crio-17a1ce1c2d881cfd8e441a4f2a0a71d3ddc86f29361eacc5620a299d4602ad58 WatchSource:0}: Error finding container 17a1ce1c2d881cfd8e441a4f2a0a71d3ddc86f29361eacc5620a299d4602ad58: Status 404 returned error can't find the container with id 17a1ce1c2d881cfd8e441a4f2a0a71d3ddc86f29361eacc5620a299d4602ad58 Jun 03 16:53:07.177008 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.176972 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-849d86d84c-522pl"] Jun 03 16:53:07.181501 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.181485 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.191451 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.191432 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 03 16:53:07.197442 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.196723 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:53:07.197442 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.196738 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 03 16:53:07.202373 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.199283 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 03 16:53:07.204738 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.204714 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 03 16:53:07.204985 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.204965 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c8ncx\"" Jun 03 16:53:07.217605 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.217586 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849d86d84c-522pl"] Jun 03 16:53:07.313141 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.313110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3394bb26-7eda-4c45-912c-d859d3b1859a-metrics-cert\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.313286 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.313152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3394bb26-7eda-4c45-912c-d859d3b1859a-cert\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.313286 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.313265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dq5\" (UniqueName: \"kubernetes.io/projected/3394bb26-7eda-4c45-912c-d859d3b1859a-kube-api-access-r9dq5\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.313387 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.313309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3394bb26-7eda-4c45-912c-d859d3b1859a-manager-config\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.414159 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.414126 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3394bb26-7eda-4c45-912c-d859d3b1859a-cert\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.414290 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.414193 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dq5\" (UniqueName: \"kubernetes.io/projected/3394bb26-7eda-4c45-912c-d859d3b1859a-kube-api-access-r9dq5\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.414290 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.414226 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3394bb26-7eda-4c45-912c-d859d3b1859a-manager-config\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.414400 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.414290 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3394bb26-7eda-4c45-912c-d859d3b1859a-metrics-cert\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.414869 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.414849 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3394bb26-7eda-4c45-912c-d859d3b1859a-manager-config\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.416505 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.416486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3394bb26-7eda-4c45-912c-d859d3b1859a-metrics-cert\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.416600 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.416540 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3394bb26-7eda-4c45-912c-d859d3b1859a-cert\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.423540 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.423521 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dq5\" (UniqueName: \"kubernetes.io/projected/3394bb26-7eda-4c45-912c-d859d3b1859a-kube-api-access-r9dq5\") pod \"lws-controller-manager-849d86d84c-522pl\" (UID: \"3394bb26-7eda-4c45-912c-d859d3b1859a\") " pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.490682 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.490631 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:07.657653 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.657627 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849d86d84c-522pl"] Jun 03 16:53:07.660246 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:53:07.660212 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3394bb26_7eda_4c45_912c_d859d3b1859a.slice/crio-0642599f651e5197727759658d6d71dd4b295a88e4702c4880a0c3a507f62d8a WatchSource:0}: Error finding container 0642599f651e5197727759658d6d71dd4b295a88e4702c4880a0c3a507f62d8a: Status 404 returned error can't find the container with id 0642599f651e5197727759658d6d71dd4b295a88e4702c4880a0c3a507f62d8a Jun 03 16:53:07.883272 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.883223 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" event={"ID":"eccef2f7-2352-415f-8eda-cee467d6b719","Type":"ContainerStarted","Data":"17a1ce1c2d881cfd8e441a4f2a0a71d3ddc86f29361eacc5620a299d4602ad58"} Jun 03 16:53:07.884572 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:07.884521 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" event={"ID":"3394bb26-7eda-4c45-912c-d859d3b1859a","Type":"ContainerStarted","Data":"0642599f651e5197727759658d6d71dd4b295a88e4702c4880a0c3a507f62d8a"} Jun 03 16:53:10.896711 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:10.896677 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" event={"ID":"3394bb26-7eda-4c45-912c-d859d3b1859a","Type":"ContainerStarted","Data":"9f2831072b93433854e129b60fa96b27c4a54e4fe33e0276616198846c1e23d5"} Jun 03 16:53:10.897119 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:10.896761 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:10.898067 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:10.898047 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" event={"ID":"eccef2f7-2352-415f-8eda-cee467d6b719","Type":"ContainerStarted","Data":"8eaaa960e57294f499c81f9311a3d8a3711592fad76e478ba28c10c307d0b101"} Jun 03 16:53:10.921142 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:10.921098 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" podStartSLOduration=1.165763385 podStartE2EDuration="3.921084817s" podCreationTimestamp="2026-06-03 16:53:07 +0000 UTC" firstStartedPulling="2026-06-03 16:53:07.662860117 +0000 UTC m=+560.046207517" lastFinishedPulling="2026-06-03 16:53:10.418181534 +0000 UTC m=+562.801528949" observedRunningTime="2026-06-03 16:53:10.920107093 +0000 UTC m=+563.303454511" watchObservedRunningTime="2026-06-03 16:53:10.921084817 +0000 UTC m=+563.304432235" Jun 03 16:53:11.903465 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:11.903428 2578 generic.go:358] "Generic (PLEG): container finished" podID="eccef2f7-2352-415f-8eda-cee467d6b719" containerID="8eaaa960e57294f499c81f9311a3d8a3711592fad76e478ba28c10c307d0b101" exitCode=0 Jun 03 16:53:11.903937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:11.903515 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" event={"ID":"eccef2f7-2352-415f-8eda-cee467d6b719","Type":"ContainerDied","Data":"8eaaa960e57294f499c81f9311a3d8a3711592fad76e478ba28c10c307d0b101"} Jun 03 16:53:12.908415 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:12.908376 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" event={"ID":"eccef2f7-2352-415f-8eda-cee467d6b719","Type":"ContainerStarted","Data":"b6929512439ab15ea44047eca330311e8bb7725dbf1ceed5e7f424a58dd4d8da"} Jun 03 16:53:12.908814 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:12.908448 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:12.925135 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:12.925059 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" podStartSLOduration=1.702895292 podStartE2EDuration="6.925042624s" podCreationTimestamp="2026-06-03 16:53:06 +0000 UTC" firstStartedPulling="2026-06-03 16:53:07.007442869 +0000 UTC m=+559.390790266" lastFinishedPulling="2026-06-03 16:53:12.229590197 +0000 UTC m=+564.612937598" observedRunningTime="2026-06-03 16:53:12.924438202 +0000 UTC m=+565.307785620" watchObservedRunningTime="2026-06-03 16:53:12.925042624 +0000 UTC m=+565.308390041" Jun 03 16:53:21.905692 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:21.905663 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-849d86d84c-522pl" Jun 03 16:53:23.914677 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:23.914648 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d99597487-4jph6" Jun 03 16:53:27.029719 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.029685 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg"] Jun 03 16:53:27.037802 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.037779 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.039883 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.039856 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 03 16:53:27.040157 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.040141 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 03 16:53:27.040444 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.040424 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mrll8\"" Jun 03 16:53:27.040596 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.040552 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 16:53:27.040713 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.040673 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 16:53:27.041614 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.041592 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg"] Jun 03 16:53:27.167274 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.167244 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbq6v\" (UniqueName: \"kubernetes.io/projected/7c022254-cdbc-4a0f-8665-ddca57b5ba71-kube-api-access-zbq6v\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.167424 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.167290 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c022254-cdbc-4a0f-8665-ddca57b5ba71-tls-certs\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.167424 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.167321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c022254-cdbc-4a0f-8665-ddca57b5ba71-tmp\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.260111 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.260078 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x"] Jun 03 16:53:27.263617 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.263600 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.265910 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.265885 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:53:27.266182 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.266166 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:53:27.266701 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.266683 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2wd4b\"" Jun 03 16:53:27.267732 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.267712 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c022254-cdbc-4a0f-8665-ddca57b5ba71-tls-certs\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.267833 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.267747 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c022254-cdbc-4a0f-8665-ddca57b5ba71-tmp\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.267833 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.267828 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbq6v\" (UniqueName: \"kubernetes.io/projected/7c022254-cdbc-4a0f-8665-ddca57b5ba71-kube-api-access-zbq6v\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.270333 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.270236 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c022254-cdbc-4a0f-8665-ddca57b5ba71-tmp\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.270427 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.270386 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c022254-cdbc-4a0f-8665-ddca57b5ba71-tls-certs\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.275920 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.275897 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x"] Jun 03 16:53:27.278545 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.278514 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbq6v\" (UniqueName: \"kubernetes.io/projected/7c022254-cdbc-4a0f-8665-ddca57b5ba71-kube-api-access-zbq6v\") pod \"kube-auth-proxy-8f98dcdbc-xfbkg\" (UID: \"7c022254-cdbc-4a0f-8665-ddca57b5ba71\") " pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.349483 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.349456 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" Jun 03 16:53:27.369143 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.369118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.369260 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.369164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.369260 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.369246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m6vl\" (UniqueName: \"kubernetes.io/projected/1999dd05-b4f0-48e3-8673-18e55f315d92-kube-api-access-7m6vl\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.470337 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.470301 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.470470 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.470344 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.470470 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.470380 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m6vl\" (UniqueName: \"kubernetes.io/projected/1999dd05-b4f0-48e3-8673-18e55f315d92-kube-api-access-7m6vl\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.470765 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.470742 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.470807 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.470756 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.473978 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.473936 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg"] Jun 03 16:53:27.476833 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:53:27.476808 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c022254_cdbc_4a0f_8665_ddca57b5ba71.slice/crio-a2690b67ccb8547b1ef12572d793aebd401fe805d29996a1af4d5caaf2082ce1 WatchSource:0}: Error finding container a2690b67ccb8547b1ef12572d793aebd401fe805d29996a1af4d5caaf2082ce1: Status 404 returned error can't find the container with id a2690b67ccb8547b1ef12572d793aebd401fe805d29996a1af4d5caaf2082ce1 Jun 03 16:53:27.478950 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.478931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m6vl\" (UniqueName: \"kubernetes.io/projected/1999dd05-b4f0-48e3-8673-18e55f315d92-kube-api-access-7m6vl\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.588237 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.588207 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:27.721133 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.721100 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x"] Jun 03 16:53:27.724165 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:53:27.724139 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1999dd05_b4f0_48e3_8673_18e55f315d92.slice/crio-7c2e6f267952a07cf5a96c1dd7601931dfbe2164a7872c4bb36fcf9768b66275 WatchSource:0}: Error finding container 7c2e6f267952a07cf5a96c1dd7601931dfbe2164a7872c4bb36fcf9768b66275: Status 404 returned error can't find the container with id 7c2e6f267952a07cf5a96c1dd7601931dfbe2164a7872c4bb36fcf9768b66275 Jun 03 16:53:27.959916 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.959832 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" event={"ID":"7c022254-cdbc-4a0f-8665-ddca57b5ba71","Type":"ContainerStarted","Data":"a2690b67ccb8547b1ef12572d793aebd401fe805d29996a1af4d5caaf2082ce1"} Jun 03 16:53:27.961195 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.961171 2578 generic.go:358] "Generic (PLEG): container finished" podID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerID="42106b2f5ad0acc68294feb9ce0e1dbc2175aaed80202dd783693206f9ca8c9f" exitCode=0 Jun 03 16:53:27.961294 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.961203 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" event={"ID":"1999dd05-b4f0-48e3-8673-18e55f315d92","Type":"ContainerDied","Data":"42106b2f5ad0acc68294feb9ce0e1dbc2175aaed80202dd783693206f9ca8c9f"} Jun 03 16:53:27.961294 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:27.961220 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" event={"ID":"1999dd05-b4f0-48e3-8673-18e55f315d92","Type":"ContainerStarted","Data":"7c2e6f267952a07cf5a96c1dd7601931dfbe2164a7872c4bb36fcf9768b66275"} Jun 03 16:53:28.974103 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:28.974029 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" event={"ID":"1999dd05-b4f0-48e3-8673-18e55f315d92","Type":"ContainerStarted","Data":"ce4039d3e138d5f120cbaef0ce50365689ad51cdc75493bc3e70ed0d43c2a60a"} Jun 03 16:53:30.983288 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:30.983236 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" event={"ID":"7c022254-cdbc-4a0f-8665-ddca57b5ba71","Type":"ContainerStarted","Data":"54817d6f4fa5aba59594119fcf9a1551ae9e59f5049b19b3b4ca968056b20d29"} Jun 03 16:53:30.984926 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:30.984888 2578 generic.go:358] "Generic (PLEG): container finished" podID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerID="ce4039d3e138d5f120cbaef0ce50365689ad51cdc75493bc3e70ed0d43c2a60a" exitCode=0 Jun 03 16:53:30.985057 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:30.984926 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" event={"ID":"1999dd05-b4f0-48e3-8673-18e55f315d92","Type":"ContainerDied","Data":"ce4039d3e138d5f120cbaef0ce50365689ad51cdc75493bc3e70ed0d43c2a60a"} Jun 03 16:53:30.999795 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:30.999744 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8f98dcdbc-xfbkg" podStartSLOduration=1.265153298 podStartE2EDuration="3.999727631s" podCreationTimestamp="2026-06-03 16:53:27 +0000 UTC" firstStartedPulling="2026-06-03 16:53:27.478835135 +0000 UTC m=+579.862182532" lastFinishedPulling="2026-06-03 16:53:30.213409469 +0000 UTC m=+582.596756865" observedRunningTime="2026-06-03 16:53:30.99913784 +0000 UTC m=+583.382485259" watchObservedRunningTime="2026-06-03 16:53:30.999727631 +0000 UTC m=+583.383075068" Jun 03 16:53:31.990037 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:31.990006 2578 generic.go:358] "Generic (PLEG): container finished" podID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerID="9bc46c8241387090a3573bac78576a9c23daba5a81519571cb340ba972de64cc" exitCode=0 Jun 03 16:53:31.990413 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:31.990077 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" event={"ID":"1999dd05-b4f0-48e3-8673-18e55f315d92","Type":"ContainerDied","Data":"9bc46c8241387090a3573bac78576a9c23daba5a81519571cb340ba972de64cc"} Jun 03 16:53:33.114202 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.114182 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:33.213869 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.213841 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-bundle\") pod \"1999dd05-b4f0-48e3-8673-18e55f315d92\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " Jun 03 16:53:33.214016 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.213890 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m6vl\" (UniqueName: \"kubernetes.io/projected/1999dd05-b4f0-48e3-8673-18e55f315d92-kube-api-access-7m6vl\") pod \"1999dd05-b4f0-48e3-8673-18e55f315d92\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " Jun 03 16:53:33.214016 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.213929 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-util\") pod \"1999dd05-b4f0-48e3-8673-18e55f315d92\" (UID: \"1999dd05-b4f0-48e3-8673-18e55f315d92\") " Jun 03 16:53:33.214789 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.214764 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-bundle" (OuterVolumeSpecName: "bundle") pod "1999dd05-b4f0-48e3-8673-18e55f315d92" (UID: "1999dd05-b4f0-48e3-8673-18e55f315d92"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:53:33.215981 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.215953 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1999dd05-b4f0-48e3-8673-18e55f315d92-kube-api-access-7m6vl" (OuterVolumeSpecName: "kube-api-access-7m6vl") pod "1999dd05-b4f0-48e3-8673-18e55f315d92" (UID: "1999dd05-b4f0-48e3-8673-18e55f315d92"). InnerVolumeSpecName "kube-api-access-7m6vl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:53:33.314625 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.314528 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-util" (OuterVolumeSpecName: "util") pod "1999dd05-b4f0-48e3-8673-18e55f315d92" (UID: "1999dd05-b4f0-48e3-8673-18e55f315d92"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:53:33.314934 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.314911 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m6vl\" (UniqueName: \"kubernetes.io/projected/1999dd05-b4f0-48e3-8673-18e55f315d92-kube-api-access-7m6vl\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:53:33.315008 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.314939 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:53:33.315008 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.314955 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1999dd05-b4f0-48e3-8673-18e55f315d92-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:53:33.998139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.998105 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" event={"ID":"1999dd05-b4f0-48e3-8673-18e55f315d92","Type":"ContainerDied","Data":"7c2e6f267952a07cf5a96c1dd7601931dfbe2164a7872c4bb36fcf9768b66275"} Jun 03 16:53:33.998139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.998121 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835v264x" Jun 03 16:53:33.998139 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:33.998136 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c2e6f267952a07cf5a96c1dd7601931dfbe2164a7872c4bb36fcf9768b66275" Jun 03 16:53:41.642476 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642432 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg"] Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642757 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="pull" Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642768 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="pull" Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642784 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="extract" Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642790 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="extract" Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642800 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="util" Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642805 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="util" Jun 03 16:53:41.642921 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.642864 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="1999dd05-b4f0-48e3-8673-18e55f315d92" containerName="extract" Jun 03 16:53:41.653138 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.653108 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.656490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.656464 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2wd4b\"" Jun 03 16:53:41.656850 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.656828 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:53:41.657245 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.657225 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:53:41.667875 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.667855 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg"] Jun 03 16:53:41.676004 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.675983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h4mp\" (UniqueName: \"kubernetes.io/projected/8fe080de-0dd4-4cff-97a0-31d584771f2c-kube-api-access-4h4mp\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.676120 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.676086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.676171 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.676155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.776944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.776909 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4h4mp\" (UniqueName: \"kubernetes.io/projected/8fe080de-0dd4-4cff-97a0-31d584771f2c-kube-api-access-4h4mp\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.777103 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.776985 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.777103 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.777023 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.777390 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.777368 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.777465 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.777442 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.796160 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.796128 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h4mp\" (UniqueName: \"kubernetes.io/projected/8fe080de-0dd4-4cff-97a0-31d584771f2c-kube-api-access-4h4mp\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:41.963777 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:41.963694 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:42.100841 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:42.100816 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg"] Jun 03 16:53:42.101866 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:53:42.101839 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe080de_0dd4_4cff_97a0_31d584771f2c.slice/crio-ae122e855ae32af5eedd540797870bc17194f3e17ecad4c9199fbbffaf32bc17 WatchSource:0}: Error finding container ae122e855ae32af5eedd540797870bc17194f3e17ecad4c9199fbbffaf32bc17: Status 404 returned error can't find the container with id ae122e855ae32af5eedd540797870bc17194f3e17ecad4c9199fbbffaf32bc17 Jun 03 16:53:43.032110 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:43.032076 2578 generic.go:358] "Generic (PLEG): container finished" podID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerID="fc3cba837536a8c4ec77365af6b7da885d9173f8cea3d874110c3ec52b725365" exitCode=0 Jun 03 16:53:43.032110 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:43.032110 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" event={"ID":"8fe080de-0dd4-4cff-97a0-31d584771f2c","Type":"ContainerDied","Data":"fc3cba837536a8c4ec77365af6b7da885d9173f8cea3d874110c3ec52b725365"} Jun 03 16:53:43.032657 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:43.032130 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" event={"ID":"8fe080de-0dd4-4cff-97a0-31d584771f2c","Type":"ContainerStarted","Data":"ae122e855ae32af5eedd540797870bc17194f3e17ecad4c9199fbbffaf32bc17"} Jun 03 16:53:45.039780 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:45.039746 2578 generic.go:358] "Generic (PLEG): container finished" podID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerID="90f6e25ad9ada726736146396bb217b3e8ebe517249732f2d57a7ec0f042cc2d" exitCode=0 Jun 03 16:53:45.040153 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:45.039798 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" event={"ID":"8fe080de-0dd4-4cff-97a0-31d584771f2c","Type":"ContainerDied","Data":"90f6e25ad9ada726736146396bb217b3e8ebe517249732f2d57a7ec0f042cc2d"} Jun 03 16:53:46.045131 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:46.045096 2578 generic.go:358] "Generic (PLEG): container finished" podID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerID="6b33f0093d38982ea65fbcea9d156e5b98ebdf90deb5aa5162f3112a5ca7160d" exitCode=0 Jun 03 16:53:46.045509 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:46.045186 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" event={"ID":"8fe080de-0dd4-4cff-97a0-31d584771f2c","Type":"ContainerDied","Data":"6b33f0093d38982ea65fbcea9d156e5b98ebdf90deb5aa5162f3112a5ca7160d"} Jun 03 16:53:47.169979 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.169949 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:53:47.221278 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.221252 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-bundle\") pod \"8fe080de-0dd4-4cff-97a0-31d584771f2c\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " Jun 03 16:53:47.221391 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.221294 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h4mp\" (UniqueName: \"kubernetes.io/projected/8fe080de-0dd4-4cff-97a0-31d584771f2c-kube-api-access-4h4mp\") pod \"8fe080de-0dd4-4cff-97a0-31d584771f2c\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " Jun 03 16:53:47.221391 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.221335 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-util\") pod \"8fe080de-0dd4-4cff-97a0-31d584771f2c\" (UID: \"8fe080de-0dd4-4cff-97a0-31d584771f2c\") " Jun 03 16:53:47.222179 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.222140 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-bundle" (OuterVolumeSpecName: "bundle") pod "8fe080de-0dd4-4cff-97a0-31d584771f2c" (UID: "8fe080de-0dd4-4cff-97a0-31d584771f2c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:53:47.223302 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.223279 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fe080de-0dd4-4cff-97a0-31d584771f2c-kube-api-access-4h4mp" (OuterVolumeSpecName: "kube-api-access-4h4mp") pod "8fe080de-0dd4-4cff-97a0-31d584771f2c" (UID: "8fe080de-0dd4-4cff-97a0-31d584771f2c"). InnerVolumeSpecName "kube-api-access-4h4mp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:53:47.227330 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.227297 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-util" (OuterVolumeSpecName: "util") pod "8fe080de-0dd4-4cff-97a0-31d584771f2c" (UID: "8fe080de-0dd4-4cff-97a0-31d584771f2c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:53:47.322318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.322258 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:53:47.322318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.322284 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h4mp\" (UniqueName: \"kubernetes.io/projected/8fe080de-0dd4-4cff-97a0-31d584771f2c-kube-api-access-4h4mp\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:53:47.322318 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:47.322295 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fe080de-0dd4-4cff-97a0-31d584771f2c-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:53:48.033030 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:48.033001 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:53:48.033181 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:48.033160 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:53:48.053057 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:48.053026 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" event={"ID":"8fe080de-0dd4-4cff-97a0-31d584771f2c","Type":"ContainerDied","Data":"ae122e855ae32af5eedd540797870bc17194f3e17ecad4c9199fbbffaf32bc17"} Jun 03 16:53:48.053149 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:48.053062 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae122e855ae32af5eedd540797870bc17194f3e17ecad4c9199fbbffaf32bc17" Jun 03 16:53:48.053149 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:53:48.053035 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ahhbjg" Jun 03 16:54:40.191373 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191342 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm"] Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191665 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="pull" Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191676 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="pull" Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191692 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="util" Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191698 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="util" Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191708 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="extract" Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191713 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="extract" Jun 03 16:54:40.191832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.191763 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="8fe080de-0dd4-4cff-97a0-31d584771f2c" containerName="extract" Jun 03 16:54:40.194856 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.194840 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.197720 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.197698 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:54:40.197846 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.197796 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lz6p5\"" Jun 03 16:54:40.198360 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.198345 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:54:40.205957 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.205939 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm"] Jun 03 16:54:40.334166 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.334129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqbhv\" (UniqueName: \"kubernetes.io/projected/ff83b8fb-6fb8-458a-947d-6a808890cf12-kube-api-access-mqbhv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.334343 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.334226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.334343 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.334244 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.440534 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.435336 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.440534 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.435391 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.440534 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.435438 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mqbhv\" (UniqueName: \"kubernetes.io/projected/ff83b8fb-6fb8-458a-947d-6a808890cf12-kube-api-access-mqbhv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.440534 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.436294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.440534 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.436587 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.446670 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.446614 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqbhv\" (UniqueName: \"kubernetes.io/projected/ff83b8fb-6fb8-458a-947d-6a808890cf12-kube-api-access-mqbhv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.504343 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.504305 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:40.639910 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.639881 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm"] Jun 03 16:54:40.641884 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:54:40.641861 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff83b8fb_6fb8_458a_947d_6a808890cf12.slice/crio-eb9cdd460055d5836020d65b54f33eae4a93e56b2d1a0b940230c467708d2d64 WatchSource:0}: Error finding container eb9cdd460055d5836020d65b54f33eae4a93e56b2d1a0b940230c467708d2d64: Status 404 returned error can't find the container with id eb9cdd460055d5836020d65b54f33eae4a93e56b2d1a0b940230c467708d2d64 Jun 03 16:54:40.804461 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.804435 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds"] Jun 03 16:54:40.808028 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.808011 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.823859 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.823838 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds"] Jun 03 16:54:40.839199 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.839175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.839290 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.839209 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjwrv\" (UniqueName: \"kubernetes.io/projected/a683e7eb-2577-4308-a28e-00bbaf70850e-kube-api-access-xjwrv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.839290 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.839230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.939756 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.939727 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.939870 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.939762 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjwrv\" (UniqueName: \"kubernetes.io/projected/a683e7eb-2577-4308-a28e-00bbaf70850e-kube-api-access-xjwrv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.939870 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.939782 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.940107 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.940084 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.940175 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.940124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:40.950731 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:40.950709 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjwrv\" (UniqueName: \"kubernetes.io/projected/a683e7eb-2577-4308-a28e-00bbaf70850e-kube-api-access-xjwrv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:41.121452 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.121426 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:41.234473 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.234435 2578 generic.go:358] "Generic (PLEG): container finished" podID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerID="8ffda370656d9125388e697f34d4c96cb5625c44d464cd6c54b27d1d3881f830" exitCode=0 Jun 03 16:54:41.234836 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.234497 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" event={"ID":"ff83b8fb-6fb8-458a-947d-6a808890cf12","Type":"ContainerDied","Data":"8ffda370656d9125388e697f34d4c96cb5625c44d464cd6c54b27d1d3881f830"} Jun 03 16:54:41.234836 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.234525 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" event={"ID":"ff83b8fb-6fb8-458a-947d-6a808890cf12","Type":"ContainerStarted","Data":"eb9cdd460055d5836020d65b54f33eae4a93e56b2d1a0b940230c467708d2d64"} Jun 03 16:54:41.245199 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.245175 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds"] Jun 03 16:54:41.246655 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:54:41.246631 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda683e7eb_2577_4308_a28e_00bbaf70850e.slice/crio-d9585665d1aa85c9d49d0f139d3b634ad5c23cc1840326202faa4c9b88620952 WatchSource:0}: Error finding container d9585665d1aa85c9d49d0f139d3b634ad5c23cc1840326202faa4c9b88620952: Status 404 returned error can't find the container with id d9585665d1aa85c9d49d0f139d3b634ad5c23cc1840326202faa4c9b88620952 Jun 03 16:54:41.410126 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.410068 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk"] Jun 03 16:54:41.413532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.413513 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.434094 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.434072 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk"] Jun 03 16:54:41.443626 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.443602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.443703 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.443661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.443703 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.443681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z7bc\" (UniqueName: \"kubernetes.io/projected/f7f4efc1-432d-4a59-bd54-09b98b89decf-kube-api-access-5z7bc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.544404 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.544379 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.544489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.544414 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5z7bc\" (UniqueName: \"kubernetes.io/projected/f7f4efc1-432d-4a59-bd54-09b98b89decf-kube-api-access-5z7bc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.544489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.544477 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.544817 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.544798 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.544876 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.544822 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.556013 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.555987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z7bc\" (UniqueName: \"kubernetes.io/projected/f7f4efc1-432d-4a59-bd54-09b98b89decf-kube-api-access-5z7bc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.733057 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.732997 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:41.860755 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:41.860727 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk"] Jun 03 16:54:41.862375 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:54:41.862346 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f4efc1_432d_4a59_bd54_09b98b89decf.slice/crio-5b028ce7144f33f7e437060a0951d6a617703a3e77dfc509a4a5275600a32ce4 WatchSource:0}: Error finding container 5b028ce7144f33f7e437060a0951d6a617703a3e77dfc509a4a5275600a32ce4: Status 404 returned error can't find the container with id 5b028ce7144f33f7e437060a0951d6a617703a3e77dfc509a4a5275600a32ce4 Jun 03 16:54:42.012371 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.012304 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8"] Jun 03 16:54:42.016017 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.015991 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.029193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.029170 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8"] Jun 03 16:54:42.049279 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.049246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.049390 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.049287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.049390 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.049349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdzlm\" (UniqueName: \"kubernetes.io/projected/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-kube-api-access-vdzlm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.149813 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.149784 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.149942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.149842 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.149942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.149887 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vdzlm\" (UniqueName: \"kubernetes.io/projected/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-kube-api-access-vdzlm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.150383 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.150358 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.150464 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.150366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.158626 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.158604 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdzlm\" (UniqueName: \"kubernetes.io/projected/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-kube-api-access-vdzlm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.239010 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.238977 2578 generic.go:358] "Generic (PLEG): container finished" podID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerID="9a10f314ad436bd256b7d1b8711b50f89f5a485b8dd7743fdbfec946bc4913b1" exitCode=0 Jun 03 16:54:42.239423 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.239049 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" event={"ID":"ff83b8fb-6fb8-458a-947d-6a808890cf12","Type":"ContainerDied","Data":"9a10f314ad436bd256b7d1b8711b50f89f5a485b8dd7743fdbfec946bc4913b1"} Jun 03 16:54:42.240393 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.240355 2578 generic.go:358] "Generic (PLEG): container finished" podID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerID="7b947cccb133cac2d496e13ec903dba731ff38c7e021d72a9e6b66d49e1831eb" exitCode=0 Jun 03 16:54:42.240489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.240392 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" event={"ID":"a683e7eb-2577-4308-a28e-00bbaf70850e","Type":"ContainerDied","Data":"7b947cccb133cac2d496e13ec903dba731ff38c7e021d72a9e6b66d49e1831eb"} Jun 03 16:54:42.240489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.240420 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" event={"ID":"a683e7eb-2577-4308-a28e-00bbaf70850e","Type":"ContainerStarted","Data":"d9585665d1aa85c9d49d0f139d3b634ad5c23cc1840326202faa4c9b88620952"} Jun 03 16:54:42.241905 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.241886 2578 generic.go:358] "Generic (PLEG): container finished" podID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerID="074738c117f3672dfd1f31b82641508d7696f44ff73c7722b5cd04c4cce9bef8" exitCode=0 Jun 03 16:54:42.242000 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.241973 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" event={"ID":"f7f4efc1-432d-4a59-bd54-09b98b89decf","Type":"ContainerDied","Data":"074738c117f3672dfd1f31b82641508d7696f44ff73c7722b5cd04c4cce9bef8"} Jun 03 16:54:42.242000 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.241994 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" event={"ID":"f7f4efc1-432d-4a59-bd54-09b98b89decf","Type":"ContainerStarted","Data":"5b028ce7144f33f7e437060a0951d6a617703a3e77dfc509a4a5275600a32ce4"} Jun 03 16:54:42.326939 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.326913 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:42.457093 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:42.457040 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8"] Jun 03 16:54:42.461530 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:54:42.461503 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95d1e32_a7e9_4673_aa9f_f7f056a8faba.slice/crio-9b7c2aec879485de8dd0f56a8a6393739d20a5d33f3f5ff7c51fdd70959b3c94 WatchSource:0}: Error finding container 9b7c2aec879485de8dd0f56a8a6393739d20a5d33f3f5ff7c51fdd70959b3c94: Status 404 returned error can't find the container with id 9b7c2aec879485de8dd0f56a8a6393739d20a5d33f3f5ff7c51fdd70959b3c94 Jun 03 16:54:43.247326 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.247230 2578 generic.go:358] "Generic (PLEG): container finished" podID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerID="03f4e421ac18b52534c9a3e5b442ccfab441e35c1384919838f1196e6045375c" exitCode=0 Jun 03 16:54:43.247326 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.247308 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" event={"ID":"ff83b8fb-6fb8-458a-947d-6a808890cf12","Type":"ContainerDied","Data":"03f4e421ac18b52534c9a3e5b442ccfab441e35c1384919838f1196e6045375c"} Jun 03 16:54:43.248912 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.248892 2578 generic.go:358] "Generic (PLEG): container finished" podID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerID="c75b20781e7bd8bb013ba20dbd7940b53aa3edd9007edd85d992fb9ea78e6d85" exitCode=0 Jun 03 16:54:43.248998 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.248955 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" event={"ID":"a683e7eb-2577-4308-a28e-00bbaf70850e","Type":"ContainerDied","Data":"c75b20781e7bd8bb013ba20dbd7940b53aa3edd9007edd85d992fb9ea78e6d85"} Jun 03 16:54:43.250282 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.250258 2578 generic.go:358] "Generic (PLEG): container finished" podID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerID="e51d3374ffcad226ba7698d65b1ff463706cb9ade92ae60786ffb8ac1b2d824e" exitCode=0 Jun 03 16:54:43.250357 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.250334 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" event={"ID":"a95d1e32-a7e9-4673-aa9f-f7f056a8faba","Type":"ContainerDied","Data":"e51d3374ffcad226ba7698d65b1ff463706cb9ade92ae60786ffb8ac1b2d824e"} Jun 03 16:54:43.250462 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.250360 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" event={"ID":"a95d1e32-a7e9-4673-aa9f-f7f056a8faba","Type":"ContainerStarted","Data":"9b7c2aec879485de8dd0f56a8a6393739d20a5d33f3f5ff7c51fdd70959b3c94"} Jun 03 16:54:43.252153 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.252130 2578 generic.go:358] "Generic (PLEG): container finished" podID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerID="e234a0e9f1f32630e9cbff7683439b61ad63daa7248481d47daee63e99a4b3ba" exitCode=0 Jun 03 16:54:43.252250 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:43.252186 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" event={"ID":"f7f4efc1-432d-4a59-bd54-09b98b89decf","Type":"ContainerDied","Data":"e234a0e9f1f32630e9cbff7683439b61ad63daa7248481d47daee63e99a4b3ba"} Jun 03 16:54:44.257763 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.257683 2578 generic.go:358] "Generic (PLEG): container finished" podID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerID="f6438cf933c8760ab73481e5d6f6d6e1d1d07bd7fdc55ba8a214f60308e7b60d" exitCode=0 Jun 03 16:54:44.258126 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.257751 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" event={"ID":"a683e7eb-2577-4308-a28e-00bbaf70850e","Type":"ContainerDied","Data":"f6438cf933c8760ab73481e5d6f6d6e1d1d07bd7fdc55ba8a214f60308e7b60d"} Jun 03 16:54:44.259393 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.259369 2578 generic.go:358] "Generic (PLEG): container finished" podID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerID="4bc35a977ca1ba714b343f2d9640289be2c0c7f99b3233943c49ad6d9118bf6f" exitCode=0 Jun 03 16:54:44.259499 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.259447 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" event={"ID":"a95d1e32-a7e9-4673-aa9f-f7f056a8faba","Type":"ContainerDied","Data":"4bc35a977ca1ba714b343f2d9640289be2c0c7f99b3233943c49ad6d9118bf6f"} Jun 03 16:54:44.261372 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.261353 2578 generic.go:358] "Generic (PLEG): container finished" podID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerID="453ee9e18e18e14d8e3f2c7b9562a1f812ffb7a0342ab37253472e39d55c317f" exitCode=0 Jun 03 16:54:44.261451 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.261400 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" event={"ID":"f7f4efc1-432d-4a59-bd54-09b98b89decf","Type":"ContainerDied","Data":"453ee9e18e18e14d8e3f2c7b9562a1f812ffb7a0342ab37253472e39d55c317f"} Jun 03 16:54:44.390519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.390498 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:44.465568 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.465541 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-util\") pod \"ff83b8fb-6fb8-458a-947d-6a808890cf12\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " Jun 03 16:54:44.465679 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.465594 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqbhv\" (UniqueName: \"kubernetes.io/projected/ff83b8fb-6fb8-458a-947d-6a808890cf12-kube-api-access-mqbhv\") pod \"ff83b8fb-6fb8-458a-947d-6a808890cf12\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " Jun 03 16:54:44.465679 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.465616 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-bundle\") pod \"ff83b8fb-6fb8-458a-947d-6a808890cf12\" (UID: \"ff83b8fb-6fb8-458a-947d-6a808890cf12\") " Jun 03 16:54:44.466197 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.466167 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-bundle" (OuterVolumeSpecName: "bundle") pod "ff83b8fb-6fb8-458a-947d-6a808890cf12" (UID: "ff83b8fb-6fb8-458a-947d-6a808890cf12"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:44.467612 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.467593 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff83b8fb-6fb8-458a-947d-6a808890cf12-kube-api-access-mqbhv" (OuterVolumeSpecName: "kube-api-access-mqbhv") pod "ff83b8fb-6fb8-458a-947d-6a808890cf12" (UID: "ff83b8fb-6fb8-458a-947d-6a808890cf12"). InnerVolumeSpecName "kube-api-access-mqbhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:54:44.471541 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.471509 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-util" (OuterVolumeSpecName: "util") pod "ff83b8fb-6fb8-458a-947d-6a808890cf12" (UID: "ff83b8fb-6fb8-458a-947d-6a808890cf12"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:44.567173 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.567119 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqbhv\" (UniqueName: \"kubernetes.io/projected/ff83b8fb-6fb8-458a-947d-6a808890cf12-kube-api-access-mqbhv\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:44.567173 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.567144 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:44.567173 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:44.567154 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff83b8fb-6fb8-458a-947d-6a808890cf12-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:45.268013 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.267981 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" event={"ID":"ff83b8fb-6fb8-458a-947d-6a808890cf12","Type":"ContainerDied","Data":"eb9cdd460055d5836020d65b54f33eae4a93e56b2d1a0b940230c467708d2d64"} Jun 03 16:54:45.268013 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.268008 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm" Jun 03 16:54:45.268457 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.268015 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb9cdd460055d5836020d65b54f33eae4a93e56b2d1a0b940230c467708d2d64" Jun 03 16:54:45.269862 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.269837 2578 generic.go:358] "Generic (PLEG): container finished" podID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerID="3a36ee00163471b82f10b17b220a99d58d001245e0520f21c1950e808e500922" exitCode=0 Jun 03 16:54:45.270009 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.269908 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" event={"ID":"a95d1e32-a7e9-4673-aa9f-f7f056a8faba","Type":"ContainerDied","Data":"3a36ee00163471b82f10b17b220a99d58d001245e0520f21c1950e808e500922"} Jun 03 16:54:45.399435 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.399408 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:45.433807 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.433787 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:45.474095 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474069 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-bundle\") pod \"a683e7eb-2577-4308-a28e-00bbaf70850e\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " Jun 03 16:54:45.474219 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474116 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z7bc\" (UniqueName: \"kubernetes.io/projected/f7f4efc1-432d-4a59-bd54-09b98b89decf-kube-api-access-5z7bc\") pod \"f7f4efc1-432d-4a59-bd54-09b98b89decf\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " Jun 03 16:54:45.474219 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474156 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjwrv\" (UniqueName: \"kubernetes.io/projected/a683e7eb-2577-4308-a28e-00bbaf70850e-kube-api-access-xjwrv\") pod \"a683e7eb-2577-4308-a28e-00bbaf70850e\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " Jun 03 16:54:45.474219 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474185 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-bundle\") pod \"f7f4efc1-432d-4a59-bd54-09b98b89decf\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " Jun 03 16:54:45.474384 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474223 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-util\") pod \"a683e7eb-2577-4308-a28e-00bbaf70850e\" (UID: \"a683e7eb-2577-4308-a28e-00bbaf70850e\") " Jun 03 16:54:45.474384 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474250 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-util\") pod \"f7f4efc1-432d-4a59-bd54-09b98b89decf\" (UID: \"f7f4efc1-432d-4a59-bd54-09b98b89decf\") " Jun 03 16:54:45.474681 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474655 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-bundle" (OuterVolumeSpecName: "bundle") pod "f7f4efc1-432d-4a59-bd54-09b98b89decf" (UID: "f7f4efc1-432d-4a59-bd54-09b98b89decf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:45.474824 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.474778 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-bundle" (OuterVolumeSpecName: "bundle") pod "a683e7eb-2577-4308-a28e-00bbaf70850e" (UID: "a683e7eb-2577-4308-a28e-00bbaf70850e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:45.476698 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.476653 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f4efc1-432d-4a59-bd54-09b98b89decf-kube-api-access-5z7bc" (OuterVolumeSpecName: "kube-api-access-5z7bc") pod "f7f4efc1-432d-4a59-bd54-09b98b89decf" (UID: "f7f4efc1-432d-4a59-bd54-09b98b89decf"). InnerVolumeSpecName "kube-api-access-5z7bc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:54:45.477793 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.477643 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a683e7eb-2577-4308-a28e-00bbaf70850e-kube-api-access-xjwrv" (OuterVolumeSpecName: "kube-api-access-xjwrv") pod "a683e7eb-2577-4308-a28e-00bbaf70850e" (UID: "a683e7eb-2577-4308-a28e-00bbaf70850e"). InnerVolumeSpecName "kube-api-access-xjwrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:54:45.482989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.482961 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-util" (OuterVolumeSpecName: "util") pod "f7f4efc1-432d-4a59-bd54-09b98b89decf" (UID: "f7f4efc1-432d-4a59-bd54-09b98b89decf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:45.483222 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.483197 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-util" (OuterVolumeSpecName: "util") pod "a683e7eb-2577-4308-a28e-00bbaf70850e" (UID: "a683e7eb-2577-4308-a28e-00bbaf70850e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:45.575209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.575152 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:45.575209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.575176 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:45.575209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.575185 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a683e7eb-2577-4308-a28e-00bbaf70850e-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:45.575209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.575194 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z7bc\" (UniqueName: \"kubernetes.io/projected/f7f4efc1-432d-4a59-bd54-09b98b89decf-kube-api-access-5z7bc\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:45.575209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.575204 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjwrv\" (UniqueName: \"kubernetes.io/projected/a683e7eb-2577-4308-a28e-00bbaf70850e-kube-api-access-xjwrv\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:45.575209 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:45.575212 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f4efc1-432d-4a59-bd54-09b98b89decf-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:46.274964 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.274938 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" Jun 03 16:54:46.274964 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.274954 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds" event={"ID":"a683e7eb-2577-4308-a28e-00bbaf70850e","Type":"ContainerDied","Data":"d9585665d1aa85c9d49d0f139d3b634ad5c23cc1840326202faa4c9b88620952"} Jun 03 16:54:46.275393 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.274984 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9585665d1aa85c9d49d0f139d3b634ad5c23cc1840326202faa4c9b88620952" Jun 03 16:54:46.276979 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.276870 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" Jun 03 16:54:46.276979 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.276916 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk" event={"ID":"f7f4efc1-432d-4a59-bd54-09b98b89decf","Type":"ContainerDied","Data":"5b028ce7144f33f7e437060a0951d6a617703a3e77dfc509a4a5275600a32ce4"} Jun 03 16:54:46.276979 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.276940 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b028ce7144f33f7e437060a0951d6a617703a3e77dfc509a4a5275600a32ce4" Jun 03 16:54:46.409487 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.409461 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:46.483947 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.483920 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdzlm\" (UniqueName: \"kubernetes.io/projected/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-kube-api-access-vdzlm\") pod \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " Jun 03 16:54:46.484079 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.483955 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-util\") pod \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " Jun 03 16:54:46.484079 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.483995 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-bundle\") pod \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\" (UID: \"a95d1e32-a7e9-4673-aa9f-f7f056a8faba\") " Jun 03 16:54:46.484590 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.484548 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-bundle" (OuterVolumeSpecName: "bundle") pod "a95d1e32-a7e9-4673-aa9f-f7f056a8faba" (UID: "a95d1e32-a7e9-4673-aa9f-f7f056a8faba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:46.485956 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.485922 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-kube-api-access-vdzlm" (OuterVolumeSpecName: "kube-api-access-vdzlm") pod "a95d1e32-a7e9-4673-aa9f-f7f056a8faba" (UID: "a95d1e32-a7e9-4673-aa9f-f7f056a8faba"). InnerVolumeSpecName "kube-api-access-vdzlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:54:46.489193 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.489171 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-util" (OuterVolumeSpecName: "util") pod "a95d1e32-a7e9-4673-aa9f-f7f056a8faba" (UID: "a95d1e32-a7e9-4673-aa9f-f7f056a8faba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:54:46.584860 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.584835 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdzlm\" (UniqueName: \"kubernetes.io/projected/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-kube-api-access-vdzlm\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:46.584860 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.584857 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:46.584991 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:46.584867 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a95d1e32-a7e9-4673-aa9f-f7f056a8faba-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:54:47.287894 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:47.287865 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" Jun 03 16:54:47.288358 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:47.287859 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8" event={"ID":"a95d1e32-a7e9-4673-aa9f-f7f056a8faba","Type":"ContainerDied","Data":"9b7c2aec879485de8dd0f56a8a6393739d20a5d33f3f5ff7c51fdd70959b3c94"} Jun 03 16:54:47.288358 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:54:47.287980 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b7c2aec879485de8dd0f56a8a6393739d20a5d33f3f5ff7c51fdd70959b3c94" Jun 03 16:55:04.636078 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636044 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c"] Jun 03 16:55:04.636454 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636424 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="extract" Jun 03 16:55:04.636454 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636440 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="extract" Jun 03 16:55:04.636454 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636455 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="util" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636463 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="util" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636473 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="extract" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636481 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="extract" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636513 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="pull" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636522 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="pull" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636531 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="pull" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636537 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="pull" Jun 03 16:55:04.636548 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636544 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="extract" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636552 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="extract" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636625 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="util" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636634 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="util" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636644 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="util" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636649 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="util" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636658 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="pull" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636663 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="pull" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636673 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="pull" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636677 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="pull" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636683 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="util" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636687 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="util" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636697 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="extract" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636701 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="extract" Jun 03 16:55:04.636784 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636787 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="a95d1e32-a7e9-4673-aa9f-f7f056a8faba" containerName="extract" Jun 03 16:55:04.637428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636796 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="f7f4efc1-432d-4a59-bd54-09b98b89decf" containerName="extract" Jun 03 16:55:04.637428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636815 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="a683e7eb-2577-4308-a28e-00bbaf70850e" containerName="extract" Jun 03 16:55:04.637428 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.636828 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="ff83b8fb-6fb8-458a-947d-6a808890cf12" containerName="extract" Jun 03 16:55:04.639796 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.639778 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.653516 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.653491 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:55:04.656269 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.656251 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:55:04.657173 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.657156 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fs4vv\"" Jun 03 16:55:04.684539 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.684517 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c"] Jun 03 16:55:04.728636 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.728613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ede8eef1-953f-48fb-ac59-f701e934ec42-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.728740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.728688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dnq\" (UniqueName: \"kubernetes.io/projected/ede8eef1-953f-48fb-ac59-f701e934ec42-kube-api-access-28dnq\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.829885 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.829858 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ede8eef1-953f-48fb-ac59-f701e934ec42-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.829995 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.829920 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-28dnq\" (UniqueName: \"kubernetes.io/projected/ede8eef1-953f-48fb-ac59-f701e934ec42-kube-api-access-28dnq\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.830225 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.830208 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ede8eef1-953f-48fb-ac59-f701e934ec42-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.873816 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.873787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dnq\" (UniqueName: \"kubernetes.io/projected/ede8eef1-953f-48fb-ac59-f701e934ec42-kube-api-access-28dnq\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:04.951641 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:04.951577 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:05.108946 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:05.108923 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c"] Jun 03 16:55:05.111367 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:55:05.111336 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede8eef1_953f_48fb_ac59_f701e934ec42.slice/crio-e5c5a6991818eae20ebf56a151fde83797adcb0b6493a8fedea3b53790e6a9e8 WatchSource:0}: Error finding container e5c5a6991818eae20ebf56a151fde83797adcb0b6493a8fedea3b53790e6a9e8: Status 404 returned error can't find the container with id e5c5a6991818eae20ebf56a151fde83797adcb0b6493a8fedea3b53790e6a9e8 Jun 03 16:55:05.350627 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:05.350594 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" event={"ID":"ede8eef1-953f-48fb-ac59-f701e934ec42","Type":"ContainerStarted","Data":"e5c5a6991818eae20ebf56a151fde83797adcb0b6493a8fedea3b53790e6a9e8"} Jun 03 16:55:11.375098 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:11.375066 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" event={"ID":"ede8eef1-953f-48fb-ac59-f701e934ec42","Type":"ContainerStarted","Data":"2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9"} Jun 03 16:55:11.375544 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:11.375129 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:11.480888 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:11.480836 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" podStartSLOduration=1.877556174 podStartE2EDuration="7.480821566s" podCreationTimestamp="2026-06-03 16:55:04 +0000 UTC" firstStartedPulling="2026-06-03 16:55:05.113728873 +0000 UTC m=+677.497076270" lastFinishedPulling="2026-06-03 16:55:10.716994253 +0000 UTC m=+683.100341662" observedRunningTime="2026-06-03 16:55:11.479654738 +0000 UTC m=+683.863002156" watchObservedRunningTime="2026-06-03 16:55:11.480821566 +0000 UTC m=+683.864168985" Jun 03 16:55:22.380822 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:22.380792 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:24.986740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:24.986709 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c"] Jun 03 16:55:24.987118 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:24.986914 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" containerName="manager" containerID="cri-o://2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9" gracePeriod=2 Jun 03 16:55:24.999331 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:24.999303 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c"] Jun 03 16:55:25.126673 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.126645 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg"] Jun 03 16:55:25.127013 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.126996 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" containerName="manager" Jun 03 16:55:25.127013 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.127013 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" containerName="manager" Jun 03 16:55:25.127159 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.127102 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" containerName="manager" Jun 03 16:55:25.130202 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.130178 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.132994 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.132953 2578 status_manager.go:895] "Failed to get status for pod" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" is forbidden: User \"system:node:ip-10-0-136-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-251.ec2.internal' and this object" Jun 03 16:55:25.221113 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.221094 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:25.251425 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.251370 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg"] Jun 03 16:55:25.290260 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.290231 2578 status_manager.go:895] "Failed to get status for pod" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" is forbidden: User \"system:node:ip-10-0-136-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-251.ec2.internal' and this object" Jun 03 16:55:25.292528 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.292509 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28dnq\" (UniqueName: \"kubernetes.io/projected/ede8eef1-953f-48fb-ac59-f701e934ec42-kube-api-access-28dnq\") pod \"ede8eef1-953f-48fb-ac59-f701e934ec42\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " Jun 03 16:55:25.292643 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.292630 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ede8eef1-953f-48fb-ac59-f701e934ec42-extensions-socket-volume\") pod \"ede8eef1-953f-48fb-ac59-f701e934ec42\" (UID: \"ede8eef1-953f-48fb-ac59-f701e934ec42\") " Jun 03 16:55:25.292751 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.292736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9zzw\" (UniqueName: \"kubernetes.io/projected/af4188bf-6763-4d3d-a560-82534bfe955d-kube-api-access-f9zzw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-2b2jg\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.292834 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.292805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af4188bf-6763-4d3d-a560-82534bfe955d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-2b2jg\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.293083 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.293063 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede8eef1-953f-48fb-ac59-f701e934ec42-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ede8eef1-953f-48fb-ac59-f701e934ec42" (UID: "ede8eef1-953f-48fb-ac59-f701e934ec42"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:55:25.294475 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.294458 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede8eef1-953f-48fb-ac59-f701e934ec42-kube-api-access-28dnq" (OuterVolumeSpecName: "kube-api-access-28dnq") pod "ede8eef1-953f-48fb-ac59-f701e934ec42" (UID: "ede8eef1-953f-48fb-ac59-f701e934ec42"). InnerVolumeSpecName "kube-api-access-28dnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:55:25.393242 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.393211 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f9zzw\" (UniqueName: \"kubernetes.io/projected/af4188bf-6763-4d3d-a560-82534bfe955d-kube-api-access-f9zzw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-2b2jg\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.393388 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.393368 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af4188bf-6763-4d3d-a560-82534bfe955d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-2b2jg\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.393492 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.393478 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28dnq\" (UniqueName: \"kubernetes.io/projected/ede8eef1-953f-48fb-ac59-f701e934ec42-kube-api-access-28dnq\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:55:25.393541 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.393500 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ede8eef1-953f-48fb-ac59-f701e934ec42-extensions-socket-volume\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:55:25.393714 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.393698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af4188bf-6763-4d3d-a560-82534bfe955d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-2b2jg\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.424395 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.424369 2578 generic.go:358] "Generic (PLEG): container finished" podID="ede8eef1-953f-48fb-ac59-f701e934ec42" containerID="2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9" exitCode=0 Jun 03 16:55:25.424532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.424428 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" Jun 03 16:55:25.424532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.424475 2578 scope.go:117] "RemoveContainer" containerID="2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9" Jun 03 16:55:25.434590 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.434547 2578 scope.go:117] "RemoveContainer" containerID="2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9" Jun 03 16:55:25.434833 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:55:25.434816 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9\": container with ID starting with 2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9 not found: ID does not exist" containerID="2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9" Jun 03 16:55:25.434882 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.434843 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9"} err="failed to get container status \"2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9\": rpc error: code = NotFound desc = could not find container \"2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9\": container with ID starting with 2d6d461aa0cb02a13149bd08eeca73e82b261a0f5ac64227a63f25274bf7f1f9 not found: ID does not exist" Jun 03 16:55:25.439772 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.439746 2578 status_manager.go:895] "Failed to get status for pod" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" is forbidden: User \"system:node:ip-10-0-136-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-251.ec2.internal' and this object" Jun 03 16:55:25.441860 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.441842 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9zzw\" (UniqueName: \"kubernetes.io/projected/af4188bf-6763-4d3d-a560-82534bfe955d-kube-api-access-f9zzw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-2b2jg\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.443102 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.443081 2578 status_manager.go:895] "Failed to get status for pod" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-kwp2c" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-kwp2c\" is forbidden: User \"system:node:ip-10-0-136-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-251.ec2.internal' and this object" Jun 03 16:55:25.741500 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.741471 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:25.950462 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:25.950330 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg"] Jun 03 16:55:25.952970 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:55:25.952939 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4188bf_6763_4d3d_a560_82534bfe955d.slice/crio-d6d1afaa49773e7168535ecf1251ce639b0c0a6da19e75898fd23c9ba5eff6c2 WatchSource:0}: Error finding container d6d1afaa49773e7168535ecf1251ce639b0c0a6da19e75898fd23c9ba5eff6c2: Status 404 returned error can't find the container with id d6d1afaa49773e7168535ecf1251ce639b0c0a6da19e75898fd23c9ba5eff6c2 Jun 03 16:55:26.099291 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:26.099264 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ede8eef1-953f-48fb-ac59-f701e934ec42" path="/var/lib/kubelet/pods/ede8eef1-953f-48fb-ac59-f701e934ec42/volumes" Jun 03 16:55:26.429740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:26.429705 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" event={"ID":"af4188bf-6763-4d3d-a560-82534bfe955d","Type":"ContainerStarted","Data":"276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f"} Jun 03 16:55:26.429740 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:26.429744 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" event={"ID":"af4188bf-6763-4d3d-a560-82534bfe955d","Type":"ContainerStarted","Data":"d6d1afaa49773e7168535ecf1251ce639b0c0a6da19e75898fd23c9ba5eff6c2"} Jun 03 16:55:26.429994 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:26.429796 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:26.487748 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:26.487705 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" podStartSLOduration=1.487691401 podStartE2EDuration="1.487691401s" podCreationTimestamp="2026-06-03 16:55:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:55:26.486892102 +0000 UTC m=+698.870239522" watchObservedRunningTime="2026-06-03 16:55:26.487691401 +0000 UTC m=+698.871038899" Jun 03 16:55:31.270469 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.270435 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq"] Jun 03 16:55:31.273976 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.273960 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.323398 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.323374 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq"] Jun 03 16:55:31.443452 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.443420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwgh\" (UniqueName: \"kubernetes.io/projected/c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3-kube-api-access-gzwgh\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9q9jq\" (UID: \"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.443617 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.443463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9q9jq\" (UID: \"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.544888 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.544804 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwgh\" (UniqueName: \"kubernetes.io/projected/c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3-kube-api-access-gzwgh\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9q9jq\" (UID: \"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.544888 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.544852 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9q9jq\" (UID: \"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.545222 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.545201 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9q9jq\" (UID: \"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.556129 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.556098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwgh\" (UniqueName: \"kubernetes.io/projected/c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3-kube-api-access-gzwgh\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9q9jq\" (UID: \"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.584084 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.584059 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:31.763105 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:31.763077 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq"] Jun 03 16:55:31.764835 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:55:31.764806 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d6021a_9e0f_4fa9_9ce2_e35bd9d064f3.slice/crio-a86d878facab3594a66f7e1149569fb1cfb4b59d951461328816d6a8d887cd79 WatchSource:0}: Error finding container a86d878facab3594a66f7e1149569fb1cfb4b59d951461328816d6a8d887cd79: Status 404 returned error can't find the container with id a86d878facab3594a66f7e1149569fb1cfb4b59d951461328816d6a8d887cd79 Jun 03 16:55:32.456599 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:32.456501 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" event={"ID":"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3","Type":"ContainerStarted","Data":"c2bbd03ae3cc9cf8863a11a4cfd34d7de0fa62911a9e6fef6b2f17cea7de833b"} Jun 03 16:55:32.456599 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:32.456537 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" event={"ID":"c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3","Type":"ContainerStarted","Data":"a86d878facab3594a66f7e1149569fb1cfb4b59d951461328816d6a8d887cd79"} Jun 03 16:55:32.456987 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:32.456683 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:32.526627 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:32.526580 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" podStartSLOduration=1.526545977 podStartE2EDuration="1.526545977s" podCreationTimestamp="2026-06-03 16:55:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:55:32.524968317 +0000 UTC m=+704.908315735" watchObservedRunningTime="2026-06-03 16:55:32.526545977 +0000 UTC m=+704.909893395" Jun 03 16:55:37.436729 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:37.436699 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:43.462254 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.462218 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9q9jq" Jun 03 16:55:43.522179 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.522136 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg"] Jun 03 16:55:43.522453 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.522404 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" podUID="af4188bf-6763-4d3d-a560-82534bfe955d" containerName="manager" containerID="cri-o://276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f" gracePeriod=10 Jun 03 16:55:43.766292 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.766269 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:43.946489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.946463 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af4188bf-6763-4d3d-a560-82534bfe955d-extensions-socket-volume\") pod \"af4188bf-6763-4d3d-a560-82534bfe955d\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " Jun 03 16:55:43.946681 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.946517 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9zzw\" (UniqueName: \"kubernetes.io/projected/af4188bf-6763-4d3d-a560-82534bfe955d-kube-api-access-f9zzw\") pod \"af4188bf-6763-4d3d-a560-82534bfe955d\" (UID: \"af4188bf-6763-4d3d-a560-82534bfe955d\") " Jun 03 16:55:43.946837 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.946812 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4188bf-6763-4d3d-a560-82534bfe955d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af4188bf-6763-4d3d-a560-82534bfe955d" (UID: "af4188bf-6763-4d3d-a560-82534bfe955d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:55:43.948387 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:43.948365 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4188bf-6763-4d3d-a560-82534bfe955d-kube-api-access-f9zzw" (OuterVolumeSpecName: "kube-api-access-f9zzw") pod "af4188bf-6763-4d3d-a560-82534bfe955d" (UID: "af4188bf-6763-4d3d-a560-82534bfe955d"). InnerVolumeSpecName "kube-api-access-f9zzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:55:44.047956 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.047895 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9zzw\" (UniqueName: \"kubernetes.io/projected/af4188bf-6763-4d3d-a560-82534bfe955d-kube-api-access-f9zzw\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:55:44.047956 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.047927 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af4188bf-6763-4d3d-a560-82534bfe955d-extensions-socket-volume\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:55:44.501640 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.501610 2578 generic.go:358] "Generic (PLEG): container finished" podID="af4188bf-6763-4d3d-a560-82534bfe955d" containerID="276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f" exitCode=0 Jun 03 16:55:44.502015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.501674 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" Jun 03 16:55:44.502015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.501697 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" event={"ID":"af4188bf-6763-4d3d-a560-82534bfe955d","Type":"ContainerDied","Data":"276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f"} Jun 03 16:55:44.502015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.501736 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg" event={"ID":"af4188bf-6763-4d3d-a560-82534bfe955d","Type":"ContainerDied","Data":"d6d1afaa49773e7168535ecf1251ce639b0c0a6da19e75898fd23c9ba5eff6c2"} Jun 03 16:55:44.502015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.501758 2578 scope.go:117] "RemoveContainer" containerID="276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f" Jun 03 16:55:44.511977 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.511962 2578 scope.go:117] "RemoveContainer" containerID="276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f" Jun 03 16:55:44.512214 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:55:44.512197 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f\": container with ID starting with 276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f not found: ID does not exist" containerID="276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f" Jun 03 16:55:44.512254 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.512221 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f"} err="failed to get container status \"276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f\": rpc error: code = NotFound desc = could not find container \"276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f\": container with ID starting with 276cddf92b3a872212a3a781733bc286d2b5dd893cf018347a375472faefcb6f not found: ID does not exist" Jun 03 16:55:44.525237 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.525213 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg"] Jun 03 16:55:44.530797 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:44.530778 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-2b2jg"] Jun 03 16:55:46.099046 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:55:46.099013 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af4188bf-6763-4d3d-a560-82534bfe955d" path="/var/lib/kubelet/pods/af4188bf-6763-4d3d-a560-82534bfe955d/volumes" Jun 03 16:56:05.497092 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.497060 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-qcnxf"] Jun 03 16:56:05.497449 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.497400 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4188bf-6763-4d3d-a560-82534bfe955d" containerName="manager" Jun 03 16:56:05.497449 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.497411 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4188bf-6763-4d3d-a560-82534bfe955d" containerName="manager" Jun 03 16:56:05.497533 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.497474 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="af4188bf-6763-4d3d-a560-82534bfe955d" containerName="manager" Jun 03 16:56:05.500411 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.500393 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:05.504820 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.504799 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pmrvz\"" Jun 03 16:56:05.513937 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.513915 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-qcnxf"] Jun 03 16:56:05.607005 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.606977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwthd\" (UniqueName: \"kubernetes.io/projected/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1-kube-api-access-dwthd\") pod \"authorino-f99f4b5cd-qcnxf\" (UID: \"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1\") " pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:05.708312 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.708283 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dwthd\" (UniqueName: \"kubernetes.io/projected/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1-kube-api-access-dwthd\") pod \"authorino-f99f4b5cd-qcnxf\" (UID: \"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1\") " pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:05.722830 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.722802 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-2hqvw"] Jun 03 16:56:05.726085 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.726070 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:56:05.734808 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.734778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwthd\" (UniqueName: \"kubernetes.io/projected/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1-kube-api-access-dwthd\") pod \"authorino-f99f4b5cd-qcnxf\" (UID: \"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1\") " pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:05.752744 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.752691 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-2hqvw"] Jun 03 16:56:05.808751 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.808725 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:05.808962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.808943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszdv\" (UniqueName: \"kubernetes.io/projected/d50157e0-6423-42d9-afee-10076a4bc0bc-kube-api-access-rszdv\") pod \"authorino-7498df8756-2hqvw\" (UID: \"d50157e0-6423-42d9-afee-10076a4bc0bc\") " pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:56:05.910244 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.910215 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rszdv\" (UniqueName: \"kubernetes.io/projected/d50157e0-6423-42d9-afee-10076a4bc0bc-kube-api-access-rszdv\") pod \"authorino-7498df8756-2hqvw\" (UID: \"d50157e0-6423-42d9-afee-10076a4bc0bc\") " pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:56:05.919353 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.919320 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszdv\" (UniqueName: \"kubernetes.io/projected/d50157e0-6423-42d9-afee-10076a4bc0bc-kube-api-access-rszdv\") pod \"authorino-7498df8756-2hqvw\" (UID: \"d50157e0-6423-42d9-afee-10076a4bc0bc\") " pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:56:05.935370 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:05.935347 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-qcnxf"] Jun 03 16:56:05.937376 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:56:05.937341 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b999f03_a1f1_4783_b4d9_59bb3aaf79c1.slice/crio-849d5b89594ac0c81689bf524d0f6ae7cf563c88007389104e8791515933f0d1 WatchSource:0}: Error finding container 849d5b89594ac0c81689bf524d0f6ae7cf563c88007389104e8791515933f0d1: Status 404 returned error can't find the container with id 849d5b89594ac0c81689bf524d0f6ae7cf563c88007389104e8791515933f0d1 Jun 03 16:56:06.035335 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:06.035277 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:56:06.156617 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:06.156588 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-2hqvw"] Jun 03 16:56:06.158181 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:56:06.158157 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd50157e0_6423_42d9_afee_10076a4bc0bc.slice/crio-8b32f9a68ef742d6bcfb0898bc2b85d16fb90f95ad70a80b8bf8ff7a2177aef5 WatchSource:0}: Error finding container 8b32f9a68ef742d6bcfb0898bc2b85d16fb90f95ad70a80b8bf8ff7a2177aef5: Status 404 returned error can't find the container with id 8b32f9a68ef742d6bcfb0898bc2b85d16fb90f95ad70a80b8bf8ff7a2177aef5 Jun 03 16:56:06.587052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:06.586997 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2hqvw" event={"ID":"d50157e0-6423-42d9-afee-10076a4bc0bc","Type":"ContainerStarted","Data":"8b32f9a68ef742d6bcfb0898bc2b85d16fb90f95ad70a80b8bf8ff7a2177aef5"} Jun 03 16:56:06.588297 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:06.588262 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" event={"ID":"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1","Type":"ContainerStarted","Data":"849d5b89594ac0c81689bf524d0f6ae7cf563c88007389104e8791515933f0d1"} Jun 03 16:56:09.604231 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:09.604191 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2hqvw" event={"ID":"d50157e0-6423-42d9-afee-10076a4bc0bc","Type":"ContainerStarted","Data":"dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6"} Jun 03 16:56:09.606145 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:09.606105 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" event={"ID":"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1","Type":"ContainerStarted","Data":"de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b"} Jun 03 16:56:09.623754 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:09.623694 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-2hqvw" podStartSLOduration=1.340273818 podStartE2EDuration="4.623680221s" podCreationTimestamp="2026-06-03 16:56:05 +0000 UTC" firstStartedPulling="2026-06-03 16:56:06.159467757 +0000 UTC m=+738.542815168" lastFinishedPulling="2026-06-03 16:56:09.442874162 +0000 UTC m=+741.826221571" observedRunningTime="2026-06-03 16:56:09.620363848 +0000 UTC m=+742.003711268" watchObservedRunningTime="2026-06-03 16:56:09.623680221 +0000 UTC m=+742.007027640" Jun 03 16:56:09.639334 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:09.639290 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" podStartSLOduration=1.124491358 podStartE2EDuration="4.639280997s" podCreationTimestamp="2026-06-03 16:56:05 +0000 UTC" firstStartedPulling="2026-06-03 16:56:05.938730137 +0000 UTC m=+738.322077539" lastFinishedPulling="2026-06-03 16:56:09.453519776 +0000 UTC m=+741.836867178" observedRunningTime="2026-06-03 16:56:09.637413601 +0000 UTC m=+742.020761019" watchObservedRunningTime="2026-06-03 16:56:09.639280997 +0000 UTC m=+742.022628416" Jun 03 16:56:09.654343 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:09.654310 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-qcnxf"] Jun 03 16:56:11.614186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:11.614144 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" podUID="2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" containerName="authorino" containerID="cri-o://de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b" gracePeriod=30 Jun 03 16:56:11.858806 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:11.858780 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:11.962197 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:11.962115 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwthd\" (UniqueName: \"kubernetes.io/projected/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1-kube-api-access-dwthd\") pod \"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1\" (UID: \"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1\") " Jun 03 16:56:11.964113 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:11.964084 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1-kube-api-access-dwthd" (OuterVolumeSpecName: "kube-api-access-dwthd") pod "2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" (UID: "2b999f03-a1f1-4783-b4d9-59bb3aaf79c1"). InnerVolumeSpecName "kube-api-access-dwthd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:56:12.063211 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.063187 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwthd\" (UniqueName: \"kubernetes.io/projected/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1-kube-api-access-dwthd\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:56:12.618208 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.618175 2578 generic.go:358] "Generic (PLEG): container finished" podID="2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" containerID="de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b" exitCode=0 Jun 03 16:56:12.618655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.618220 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" Jun 03 16:56:12.618655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.618263 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" event={"ID":"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1","Type":"ContainerDied","Data":"de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b"} Jun 03 16:56:12.618655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.618310 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-qcnxf" event={"ID":"2b999f03-a1f1-4783-b4d9-59bb3aaf79c1","Type":"ContainerDied","Data":"849d5b89594ac0c81689bf524d0f6ae7cf563c88007389104e8791515933f0d1"} Jun 03 16:56:12.618655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.618326 2578 scope.go:117] "RemoveContainer" containerID="de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b" Jun 03 16:56:12.627027 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.626951 2578 scope.go:117] "RemoveContainer" containerID="de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b" Jun 03 16:56:12.627204 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:56:12.627187 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b\": container with ID starting with de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b not found: ID does not exist" containerID="de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b" Jun 03 16:56:12.627248 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.627213 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b"} err="failed to get container status \"de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b\": rpc error: code = NotFound desc = could not find container \"de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b\": container with ID starting with de80f4261f2c4263f1e4817ad72c75994e736dbecbaa5fd662dccdeb142a639b not found: ID does not exist" Jun 03 16:56:12.633901 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.633878 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-qcnxf"] Jun 03 16:56:12.637629 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:12.637610 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-qcnxf"] Jun 03 16:56:14.099383 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:14.099354 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" path="/var/lib/kubelet/pods/2b999f03-a1f1-4783-b4d9-59bb3aaf79c1/volumes" Jun 03 16:56:38.137372 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.137333 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc"] Jun 03 16:56:38.137878 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.137679 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" containerName="authorino" Jun 03 16:56:38.137878 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.137690 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" containerName="authorino" Jun 03 16:56:38.137878 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.137745 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b999f03-a1f1-4783-b4d9-59bb3aaf79c1" containerName="authorino" Jun 03 16:56:38.140867 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.140851 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.143589 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.143547 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2wd4b\"" Jun 03 16:56:38.143708 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.143630 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:56:38.143759 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.143749 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:56:38.152517 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.152497 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc"] Jun 03 16:56:38.278342 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.278316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.278503 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.278358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww8gz\" (UniqueName: \"kubernetes.io/projected/50dd8375-5356-4844-b217-5624da28f391-kube-api-access-ww8gz\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.278503 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.278477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.379485 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.379455 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.379662 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.379494 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.379662 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.379522 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ww8gz\" (UniqueName: \"kubernetes.io/projected/50dd8375-5356-4844-b217-5624da28f391-kube-api-access-ww8gz\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.379849 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.379829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.379885 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.379864 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.389123 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.389071 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww8gz\" (UniqueName: \"kubernetes.io/projected/50dd8375-5356-4844-b217-5624da28f391-kube-api-access-ww8gz\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.450179 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.450145 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:38.585012 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.584979 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc"] Jun 03 16:56:38.593108 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:56:38.593080 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dd8375_5356_4844_b217_5624da28f391.slice/crio-1ec328289a52c354872b1823bc8e5fd60e0634c108f3cacc3a2a67bbf5fd7fe2 WatchSource:0}: Error finding container 1ec328289a52c354872b1823bc8e5fd60e0634c108f3cacc3a2a67bbf5fd7fe2: Status 404 returned error can't find the container with id 1ec328289a52c354872b1823bc8e5fd60e0634c108f3cacc3a2a67bbf5fd7fe2 Jun 03 16:56:38.716386 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.716342 2578 generic.go:358] "Generic (PLEG): container finished" podID="50dd8375-5356-4844-b217-5624da28f391" containerID="01ba51140afb90cc2a4b33a6ac86a564ae6a851f57d1a7b2348a7c647e72225e" exitCode=0 Jun 03 16:56:38.716489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.716418 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" event={"ID":"50dd8375-5356-4844-b217-5624da28f391","Type":"ContainerDied","Data":"01ba51140afb90cc2a4b33a6ac86a564ae6a851f57d1a7b2348a7c647e72225e"} Jun 03 16:56:38.716489 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:38.716455 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" event={"ID":"50dd8375-5356-4844-b217-5624da28f391","Type":"ContainerStarted","Data":"1ec328289a52c354872b1823bc8e5fd60e0634c108f3cacc3a2a67bbf5fd7fe2"} Jun 03 16:56:39.722529 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:39.722498 2578 generic.go:358] "Generic (PLEG): container finished" podID="50dd8375-5356-4844-b217-5624da28f391" containerID="8e7fea3ebbec0a4bbcc6ae5de5b07f039b6dd6b7d6ba591497b6b266da62a294" exitCode=0 Jun 03 16:56:39.723022 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:39.722537 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" event={"ID":"50dd8375-5356-4844-b217-5624da28f391","Type":"ContainerDied","Data":"8e7fea3ebbec0a4bbcc6ae5de5b07f039b6dd6b7d6ba591497b6b266da62a294"} Jun 03 16:56:40.734048 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:40.734016 2578 generic.go:358] "Generic (PLEG): container finished" podID="50dd8375-5356-4844-b217-5624da28f391" containerID="8345f9fc48a37567149e397273910d2bb68a93089ffe6044ffd93b73fbc62691" exitCode=0 Jun 03 16:56:40.734410 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:40.734055 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" event={"ID":"50dd8375-5356-4844-b217-5624da28f391","Type":"ContainerDied","Data":"8345f9fc48a37567149e397273910d2bb68a93089ffe6044ffd93b73fbc62691"} Jun 03 16:56:41.862771 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:41.862749 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:42.010591 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.010497 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-util\") pod \"50dd8375-5356-4844-b217-5624da28f391\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " Jun 03 16:56:42.010591 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.010538 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww8gz\" (UniqueName: \"kubernetes.io/projected/50dd8375-5356-4844-b217-5624da28f391-kube-api-access-ww8gz\") pod \"50dd8375-5356-4844-b217-5624da28f391\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " Jun 03 16:56:42.010781 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.010618 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-bundle\") pod \"50dd8375-5356-4844-b217-5624da28f391\" (UID: \"50dd8375-5356-4844-b217-5624da28f391\") " Jun 03 16:56:42.011126 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.011090 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-bundle" (OuterVolumeSpecName: "bundle") pod "50dd8375-5356-4844-b217-5624da28f391" (UID: "50dd8375-5356-4844-b217-5624da28f391"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:56:42.012664 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.012639 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dd8375-5356-4844-b217-5624da28f391-kube-api-access-ww8gz" (OuterVolumeSpecName: "kube-api-access-ww8gz") pod "50dd8375-5356-4844-b217-5624da28f391" (UID: "50dd8375-5356-4844-b217-5624da28f391"). InnerVolumeSpecName "kube-api-access-ww8gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:56:42.016701 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.016666 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-util" (OuterVolumeSpecName: "util") pod "50dd8375-5356-4844-b217-5624da28f391" (UID: "50dd8375-5356-4844-b217-5624da28f391"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:56:42.111873 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.111849 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww8gz\" (UniqueName: \"kubernetes.io/projected/50dd8375-5356-4844-b217-5624da28f391-kube-api-access-ww8gz\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:56:42.111873 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.111873 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-bundle\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:56:42.112015 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.111883 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50dd8375-5356-4844-b217-5624da28f391-util\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:56:42.743624 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.743594 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" Jun 03 16:56:42.743786 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.743594 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70pt4kc" event={"ID":"50dd8375-5356-4844-b217-5624da28f391","Type":"ContainerDied","Data":"1ec328289a52c354872b1823bc8e5fd60e0634c108f3cacc3a2a67bbf5fd7fe2"} Jun 03 16:56:42.743786 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:56:42.743707 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ec328289a52c354872b1823bc8e5fd60e0634c108f3cacc3a2a67bbf5fd7fe2" Jun 03 16:57:01.926206 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926166 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926510 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="util" Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926522 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="util" Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926539 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="pull" Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926545 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="pull" Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926569 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="extract" Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926575 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="extract" Jun 03 16:57:01.926639 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.926636 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="50dd8375-5356-4844-b217-5624da28f391" containerName="extract" Jun 03 16:57:01.933202 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.933175 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:01.935205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.935176 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 03 16:57:01.935369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.935276 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-mm8wk\"" Jun 03 16:57:01.935369 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.935214 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 03 16:57:01.935648 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.935603 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 03 16:57:01.937055 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.937031 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:57:01.968722 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:01.968686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnf6f\" (UniqueName: \"kubernetes.io/projected/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3-kube-api-access-dnf6f\") pod \"maas-keycloak-0\" (UID: \"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:02.070060 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:02.070023 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dnf6f\" (UniqueName: \"kubernetes.io/projected/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3-kube-api-access-dnf6f\") pod \"maas-keycloak-0\" (UID: \"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:02.077703 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:02.077678 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnf6f\" (UniqueName: \"kubernetes.io/projected/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3-kube-api-access-dnf6f\") pod \"maas-keycloak-0\" (UID: \"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:02.243865 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:02.243754 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:02.369487 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:02.369459 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:57:02.371405 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:57:02.371380 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43dd2f4_9fc4_4283_8d20_20be0ae36bf3.slice/crio-899b1b687452e782e2cee92fa343846b49f1b1d819fdafe258809b6a8a2f3e74 WatchSource:0}: Error finding container 899b1b687452e782e2cee92fa343846b49f1b1d819fdafe258809b6a8a2f3e74: Status 404 returned error can't find the container with id 899b1b687452e782e2cee92fa343846b49f1b1d819fdafe258809b6a8a2f3e74 Jun 03 16:57:02.823749 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:02.823715 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3","Type":"ContainerStarted","Data":"899b1b687452e782e2cee92fa343846b49f1b1d819fdafe258809b6a8a2f3e74"} Jun 03 16:57:09.855130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:09.855060 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3","Type":"ContainerStarted","Data":"fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8"} Jun 03 16:57:09.872009 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:09.871954 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.262156041 podStartE2EDuration="8.871935221s" podCreationTimestamp="2026-06-03 16:57:01 +0000 UTC" firstStartedPulling="2026-06-03 16:57:02.372722326 +0000 UTC m=+794.756069723" lastFinishedPulling="2026-06-03 16:57:08.982501503 +0000 UTC m=+801.365848903" observedRunningTime="2026-06-03 16:57:09.87048884 +0000 UTC m=+802.253836259" watchObservedRunningTime="2026-06-03 16:57:09.871935221 +0000 UTC m=+802.255282643" Jun 03 16:57:10.244746 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:10.244643 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:10.246190 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:10.246150 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:11.244324 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:11.244269 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:12.244229 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:12.244169 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:12.244532 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:12.244494 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:13.244932 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:13.244861 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:14.244780 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:14.244734 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:15.244275 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:15.244230 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:16.245103 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:16.245051 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:17.244951 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:17.244904 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:18.245030 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:18.244987 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:19.244981 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:19.244922 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:20.244871 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:20.244822 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:21.245010 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:21.244955 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.46:9000/health/started\": dial tcp 10.133.0.46:9000: connect: connection refused" Jun 03 16:57:22.357835 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:22.357797 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:22.376540 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:22.376491 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:57:32.364115 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:32.364083 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 03 16:57:33.280397 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.280357 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-xhc2f"] Jun 03 16:57:33.291294 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.291266 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-xhc2f"] Jun 03 16:57:33.291441 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.291375 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:33.370016 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.369989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fhvb\" (UniqueName: \"kubernetes.io/projected/2006b820-e331-4428-9faf-975eb213dcbd-kube-api-access-5fhvb\") pod \"authorino-8b475cf9f-xhc2f\" (UID: \"2006b820-e331-4428-9faf-975eb213dcbd\") " pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:33.471158 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.471126 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5fhvb\" (UniqueName: \"kubernetes.io/projected/2006b820-e331-4428-9faf-975eb213dcbd-kube-api-access-5fhvb\") pod \"authorino-8b475cf9f-xhc2f\" (UID: \"2006b820-e331-4428-9faf-975eb213dcbd\") " pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:33.478968 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.478948 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fhvb\" (UniqueName: \"kubernetes.io/projected/2006b820-e331-4428-9faf-975eb213dcbd-kube-api-access-5fhvb\") pod \"authorino-8b475cf9f-xhc2f\" (UID: \"2006b820-e331-4428-9faf-975eb213dcbd\") " pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:33.526056 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.526033 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-xhc2f"] Jun 03 16:57:33.526253 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.526242 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:33.554087 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.554053 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c964bdbcb-888xq"] Jun 03 16:57:33.559414 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.559395 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:33.563096 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.563071 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c964bdbcb-888xq"] Jun 03 16:57:33.572096 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.572064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct462\" (UniqueName: \"kubernetes.io/projected/c3197eef-c176-4159-9023-ca301426d786-kube-api-access-ct462\") pod \"authorino-7c964bdbcb-888xq\" (UID: \"c3197eef-c176-4159-9023-ca301426d786\") " pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:33.664608 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.664573 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-xhc2f"] Jun 03 16:57:33.666704 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:57:33.666671 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2006b820_e331_4428_9faf_975eb213dcbd.slice/crio-b7936f1b542ab3409e1b30f10162b8e4d3bbaaf1262b104d86680b35030f3367 WatchSource:0}: Error finding container b7936f1b542ab3409e1b30f10162b8e4d3bbaaf1262b104d86680b35030f3367: Status 404 returned error can't find the container with id b7936f1b542ab3409e1b30f10162b8e4d3bbaaf1262b104d86680b35030f3367 Jun 03 16:57:33.668390 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.668371 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:57:33.673304 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.673279 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ct462\" (UniqueName: \"kubernetes.io/projected/c3197eef-c176-4159-9023-ca301426d786-kube-api-access-ct462\") pod \"authorino-7c964bdbcb-888xq\" (UID: \"c3197eef-c176-4159-9023-ca301426d786\") " pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:33.680851 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.680828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct462\" (UniqueName: \"kubernetes.io/projected/c3197eef-c176-4159-9023-ca301426d786-kube-api-access-ct462\") pod \"authorino-7c964bdbcb-888xq\" (UID: \"c3197eef-c176-4159-9023-ca301426d786\") " pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:33.875149 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.875128 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:33.882050 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.882026 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c964bdbcb-888xq"] Jun 03 16:57:33.913458 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.913069 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687744bd96-nmfjq"] Jun 03 16:57:33.918706 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.918683 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:33.920796 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.920772 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 03 16:57:33.923343 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.923320 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687744bd96-nmfjq"] Jun 03 16:57:33.973918 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.973878 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" event={"ID":"2006b820-e331-4428-9faf-975eb213dcbd","Type":"ContainerStarted","Data":"b7936f1b542ab3409e1b30f10162b8e4d3bbaaf1262b104d86680b35030f3367"} Jun 03 16:57:33.976274 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.976250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09f7f3d7-ec22-466c-a152-a4453e01a3bd-tls-cert\") pod \"authorino-687744bd96-nmfjq\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:33.976385 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:33.976314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcl28\" (UniqueName: \"kubernetes.io/projected/09f7f3d7-ec22-466c-a152-a4453e01a3bd-kube-api-access-qcl28\") pod \"authorino-687744bd96-nmfjq\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:34.011395 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.011374 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c964bdbcb-888xq"] Jun 03 16:57:34.012925 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:57:34.012900 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3197eef_c176_4159_9023_ca301426d786.slice/crio-0b2e22563802c67aec80df09e8a25d847e6e0fafd9fc54d2374a0b53e2617396 WatchSource:0}: Error finding container 0b2e22563802c67aec80df09e8a25d847e6e0fafd9fc54d2374a0b53e2617396: Status 404 returned error can't find the container with id 0b2e22563802c67aec80df09e8a25d847e6e0fafd9fc54d2374a0b53e2617396 Jun 03 16:57:34.076833 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.076802 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09f7f3d7-ec22-466c-a152-a4453e01a3bd-tls-cert\") pod \"authorino-687744bd96-nmfjq\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:34.076926 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.076904 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qcl28\" (UniqueName: \"kubernetes.io/projected/09f7f3d7-ec22-466c-a152-a4453e01a3bd-kube-api-access-qcl28\") pod \"authorino-687744bd96-nmfjq\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:34.079117 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.079089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09f7f3d7-ec22-466c-a152-a4453e01a3bd-tls-cert\") pod \"authorino-687744bd96-nmfjq\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:34.083763 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.083741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcl28\" (UniqueName: \"kubernetes.io/projected/09f7f3d7-ec22-466c-a152-a4453e01a3bd-kube-api-access-qcl28\") pod \"authorino-687744bd96-nmfjq\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:34.232277 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.232255 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:57:34.370526 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.370441 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687744bd96-nmfjq"] Jun 03 16:57:34.372785 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:57:34.372750 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09f7f3d7_ec22_466c_a152_a4453e01a3bd.slice/crio-40fcb3cefcd9bad3095f2e9ccfdc696b508e614999c0d4284d7fbbd65f259e45 WatchSource:0}: Error finding container 40fcb3cefcd9bad3095f2e9ccfdc696b508e614999c0d4284d7fbbd65f259e45: Status 404 returned error can't find the container with id 40fcb3cefcd9bad3095f2e9ccfdc696b508e614999c0d4284d7fbbd65f259e45 Jun 03 16:57:34.980217 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.980180 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687744bd96-nmfjq" event={"ID":"09f7f3d7-ec22-466c-a152-a4453e01a3bd","Type":"ContainerStarted","Data":"5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8"} Jun 03 16:57:34.980217 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.980221 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687744bd96-nmfjq" event={"ID":"09f7f3d7-ec22-466c-a152-a4453e01a3bd","Type":"ContainerStarted","Data":"40fcb3cefcd9bad3095f2e9ccfdc696b508e614999c0d4284d7fbbd65f259e45"} Jun 03 16:57:34.981597 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.981546 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" event={"ID":"2006b820-e331-4428-9faf-975eb213dcbd","Type":"ContainerStarted","Data":"7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2"} Jun 03 16:57:34.981597 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.981577 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" podUID="2006b820-e331-4428-9faf-975eb213dcbd" containerName="authorino" containerID="cri-o://7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2" gracePeriod=30 Jun 03 16:57:34.982942 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.982917 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c964bdbcb-888xq" event={"ID":"c3197eef-c176-4159-9023-ca301426d786","Type":"ContainerStarted","Data":"bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074"} Jun 03 16:57:34.983064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.982949 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c964bdbcb-888xq" event={"ID":"c3197eef-c176-4159-9023-ca301426d786","Type":"ContainerStarted","Data":"0b2e22563802c67aec80df09e8a25d847e6e0fafd9fc54d2374a0b53e2617396"} Jun 03 16:57:34.983064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.982919 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7c964bdbcb-888xq" podUID="c3197eef-c176-4159-9023-ca301426d786" containerName="authorino" containerID="cri-o://bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074" gracePeriod=30 Jun 03 16:57:34.994951 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:34.994869 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-687744bd96-nmfjq" podStartSLOduration=1.628899664 podStartE2EDuration="1.994857479s" podCreationTimestamp="2026-06-03 16:57:33 +0000 UTC" firstStartedPulling="2026-06-03 16:57:34.375957318 +0000 UTC m=+826.759304724" lastFinishedPulling="2026-06-03 16:57:34.741915142 +0000 UTC m=+827.125262539" observedRunningTime="2026-06-03 16:57:34.993677242 +0000 UTC m=+827.377024663" watchObservedRunningTime="2026-06-03 16:57:34.994857479 +0000 UTC m=+827.378204897" Jun 03 16:57:35.008793 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.008741 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7c964bdbcb-888xq" podStartSLOduration=1.648667391 podStartE2EDuration="2.008726975s" podCreationTimestamp="2026-06-03 16:57:33 +0000 UTC" firstStartedPulling="2026-06-03 16:57:34.014114597 +0000 UTC m=+826.397461998" lastFinishedPulling="2026-06-03 16:57:34.37417418 +0000 UTC m=+826.757521582" observedRunningTime="2026-06-03 16:57:35.006500812 +0000 UTC m=+827.389848228" watchObservedRunningTime="2026-06-03 16:57:35.008726975 +0000 UTC m=+827.392074396" Jun 03 16:57:35.023161 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.023133 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-2hqvw"] Jun 03 16:57:35.023373 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.023351 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-2hqvw" podUID="d50157e0-6423-42d9-afee-10076a4bc0bc" containerName="authorino" containerID="cri-o://dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6" gracePeriod=30 Jun 03 16:57:35.024458 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.024415 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" podStartSLOduration=1.467520545 podStartE2EDuration="2.024404462s" podCreationTimestamp="2026-06-03 16:57:33 +0000 UTC" firstStartedPulling="2026-06-03 16:57:33.668489707 +0000 UTC m=+826.051837104" lastFinishedPulling="2026-06-03 16:57:34.22537361 +0000 UTC m=+826.608721021" observedRunningTime="2026-06-03 16:57:35.019922407 +0000 UTC m=+827.403269827" watchObservedRunningTime="2026-06-03 16:57:35.024404462 +0000 UTC m=+827.407751882" Jun 03 16:57:35.277083 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.277059 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:35.288522 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.288501 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fhvb\" (UniqueName: \"kubernetes.io/projected/2006b820-e331-4428-9faf-975eb213dcbd-kube-api-access-5fhvb\") pod \"2006b820-e331-4428-9faf-975eb213dcbd\" (UID: \"2006b820-e331-4428-9faf-975eb213dcbd\") " Jun 03 16:57:35.289749 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.289730 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:35.290681 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.290657 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2006b820-e331-4428-9faf-975eb213dcbd-kube-api-access-5fhvb" (OuterVolumeSpecName: "kube-api-access-5fhvb") pod "2006b820-e331-4428-9faf-975eb213dcbd" (UID: "2006b820-e331-4428-9faf-975eb213dcbd"). InnerVolumeSpecName "kube-api-access-5fhvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:57:35.305119 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.305101 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:57:35.389527 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.389498 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rszdv\" (UniqueName: \"kubernetes.io/projected/d50157e0-6423-42d9-afee-10076a4bc0bc-kube-api-access-rszdv\") pod \"d50157e0-6423-42d9-afee-10076a4bc0bc\" (UID: \"d50157e0-6423-42d9-afee-10076a4bc0bc\") " Jun 03 16:57:35.389527 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.389530 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct462\" (UniqueName: \"kubernetes.io/projected/c3197eef-c176-4159-9023-ca301426d786-kube-api-access-ct462\") pod \"c3197eef-c176-4159-9023-ca301426d786\" (UID: \"c3197eef-c176-4159-9023-ca301426d786\") " Jun 03 16:57:35.389989 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.389741 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fhvb\" (UniqueName: \"kubernetes.io/projected/2006b820-e331-4428-9faf-975eb213dcbd-kube-api-access-5fhvb\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:57:35.391344 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.391312 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d50157e0-6423-42d9-afee-10076a4bc0bc-kube-api-access-rszdv" (OuterVolumeSpecName: "kube-api-access-rszdv") pod "d50157e0-6423-42d9-afee-10076a4bc0bc" (UID: "d50157e0-6423-42d9-afee-10076a4bc0bc"). InnerVolumeSpecName "kube-api-access-rszdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:57:35.391447 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.391383 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3197eef-c176-4159-9023-ca301426d786-kube-api-access-ct462" (OuterVolumeSpecName: "kube-api-access-ct462") pod "c3197eef-c176-4159-9023-ca301426d786" (UID: "c3197eef-c176-4159-9023-ca301426d786"). InnerVolumeSpecName "kube-api-access-ct462". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:57:35.490766 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.490743 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rszdv\" (UniqueName: \"kubernetes.io/projected/d50157e0-6423-42d9-afee-10076a4bc0bc-kube-api-access-rszdv\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:57:35.490877 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.490767 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct462\" (UniqueName: \"kubernetes.io/projected/c3197eef-c176-4159-9023-ca301426d786-kube-api-access-ct462\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:57:35.987940 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.987909 2578 generic.go:358] "Generic (PLEG): container finished" podID="2006b820-e331-4428-9faf-975eb213dcbd" containerID="7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2" exitCode=0 Jun 03 16:57:35.988137 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.987956 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" Jun 03 16:57:35.988137 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.987996 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" event={"ID":"2006b820-e331-4428-9faf-975eb213dcbd","Type":"ContainerDied","Data":"7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2"} Jun 03 16:57:35.988137 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.988034 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-xhc2f" event={"ID":"2006b820-e331-4428-9faf-975eb213dcbd","Type":"ContainerDied","Data":"b7936f1b542ab3409e1b30f10162b8e4d3bbaaf1262b104d86680b35030f3367"} Jun 03 16:57:35.988137 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.988054 2578 scope.go:117] "RemoveContainer" containerID="7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2" Jun 03 16:57:35.996186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.995905 2578 generic.go:358] "Generic (PLEG): container finished" podID="d50157e0-6423-42d9-afee-10076a4bc0bc" containerID="dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6" exitCode=0 Jun 03 16:57:35.996186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.995983 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2hqvw" event={"ID":"d50157e0-6423-42d9-afee-10076a4bc0bc","Type":"ContainerDied","Data":"dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6"} Jun 03 16:57:35.996186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.996012 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2hqvw" event={"ID":"d50157e0-6423-42d9-afee-10076a4bc0bc","Type":"ContainerDied","Data":"8b32f9a68ef742d6bcfb0898bc2b85d16fb90f95ad70a80b8bf8ff7a2177aef5"} Jun 03 16:57:35.996186 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.996098 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2hqvw" Jun 03 16:57:35.998314 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.998288 2578 generic.go:358] "Generic (PLEG): container finished" podID="c3197eef-c176-4159-9023-ca301426d786" containerID="bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074" exitCode=0 Jun 03 16:57:35.998443 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.998336 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c964bdbcb-888xq" Jun 03 16:57:35.998443 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.998368 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c964bdbcb-888xq" event={"ID":"c3197eef-c176-4159-9023-ca301426d786","Type":"ContainerDied","Data":"bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074"} Jun 03 16:57:35.998443 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:35.998396 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c964bdbcb-888xq" event={"ID":"c3197eef-c176-4159-9023-ca301426d786","Type":"ContainerDied","Data":"0b2e22563802c67aec80df09e8a25d847e6e0fafd9fc54d2374a0b53e2617396"} Jun 03 16:57:36.005733 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.005715 2578 scope.go:117] "RemoveContainer" containerID="7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2" Jun 03 16:57:36.006007 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:57:36.005987 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2\": container with ID starting with 7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2 not found: ID does not exist" containerID="7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2" Jun 03 16:57:36.006076 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.006014 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2"} err="failed to get container status \"7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2\": rpc error: code = NotFound desc = could not find container \"7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2\": container with ID starting with 7bcced065208476a2a7a7ee0d345e5962fb95a783786ffc0aa903051a159dba2 not found: ID does not exist" Jun 03 16:57:36.006076 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.006037 2578 scope.go:117] "RemoveContainer" containerID="dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6" Jun 03 16:57:36.014568 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.014535 2578 scope.go:117] "RemoveContainer" containerID="dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6" Jun 03 16:57:36.014739 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.014722 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-xhc2f"] Jun 03 16:57:36.014819 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:57:36.014800 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6\": container with ID starting with dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6 not found: ID does not exist" containerID="dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6" Jun 03 16:57:36.014864 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.014827 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6"} err="failed to get container status \"dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6\": rpc error: code = NotFound desc = could not find container \"dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6\": container with ID starting with dae39ade96b419080d76fb3a9aea0bb2024f542cb4761560c2f2895d66f1bff6 not found: ID does not exist" Jun 03 16:57:36.014864 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.014844 2578 scope.go:117] "RemoveContainer" containerID="bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074" Jun 03 16:57:36.018890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.018865 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-xhc2f"] Jun 03 16:57:36.022242 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.022227 2578 scope.go:117] "RemoveContainer" containerID="bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074" Jun 03 16:57:36.022449 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:57:36.022433 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074\": container with ID starting with bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074 not found: ID does not exist" containerID="bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074" Jun 03 16:57:36.022508 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.022452 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074"} err="failed to get container status \"bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074\": rpc error: code = NotFound desc = could not find container \"bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074\": container with ID starting with bb3c727e1af5bc087a548f22a5b5dbe404965069bee33518f60b899974a31074 not found: ID does not exist" Jun 03 16:57:36.027436 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.027417 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-2hqvw"] Jun 03 16:57:36.031484 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.031464 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-2hqvw"] Jun 03 16:57:36.040150 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.040131 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c964bdbcb-888xq"] Jun 03 16:57:36.043702 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.043682 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c964bdbcb-888xq"] Jun 03 16:57:36.098841 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.098818 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2006b820-e331-4428-9faf-975eb213dcbd" path="/var/lib/kubelet/pods/2006b820-e331-4428-9faf-975eb213dcbd/volumes" Jun 03 16:57:36.099114 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.099102 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3197eef-c176-4159-9023-ca301426d786" path="/var/lib/kubelet/pods/c3197eef-c176-4159-9023-ca301426d786/volumes" Jun 03 16:57:36.099399 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:36.099387 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d50157e0-6423-42d9-afee-10076a4bc0bc" path="/var/lib/kubelet/pods/d50157e0-6423-42d9-afee-10076a4bc0bc/volumes" Jun 03 16:57:38.107619 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.107584 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c646bcfdc-lrp24"] Jun 03 16:57:38.108052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.107977 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50157e0-6423-42d9-afee-10076a4bc0bc" containerName="authorino" Jun 03 16:57:38.108052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.107995 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50157e0-6423-42d9-afee-10076a4bc0bc" containerName="authorino" Jun 03 16:57:38.108052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108007 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3197eef-c176-4159-9023-ca301426d786" containerName="authorino" Jun 03 16:57:38.108052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108013 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3197eef-c176-4159-9023-ca301426d786" containerName="authorino" Jun 03 16:57:38.108052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108023 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2006b820-e331-4428-9faf-975eb213dcbd" containerName="authorino" Jun 03 16:57:38.108052 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108029 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2006b820-e331-4428-9faf-975eb213dcbd" containerName="authorino" Jun 03 16:57:38.108412 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108124 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="2006b820-e331-4428-9faf-975eb213dcbd" containerName="authorino" Jun 03 16:57:38.108412 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108135 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="d50157e0-6423-42d9-afee-10076a4bc0bc" containerName="authorino" Jun 03 16:57:38.108412 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.108151 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c3197eef-c176-4159-9023-ca301426d786" containerName="authorino" Jun 03 16:57:38.112716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.112700 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:38.114796 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.114776 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 03 16:57:38.114879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.114816 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n7rcl\"" Jun 03 16:57:38.122543 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.122520 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c646bcfdc-lrp24"] Jun 03 16:57:38.213229 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.213201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8trj\" (UniqueName: \"kubernetes.io/projected/0f093c3c-bac1-4564-91fd-cf382892a875-kube-api-access-x8trj\") pod \"maas-controller-7c646bcfdc-lrp24\" (UID: \"0f093c3c-bac1-4564-91fd-cf382892a875\") " pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:38.271107 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.271082 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68c5cd4545-5k6cj"] Jun 03 16:57:38.274332 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.274313 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:38.281090 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.281064 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c5cd4545-5k6cj"] Jun 03 16:57:38.313856 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.313830 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x8trj\" (UniqueName: \"kubernetes.io/projected/0f093c3c-bac1-4564-91fd-cf382892a875-kube-api-access-x8trj\") pod \"maas-controller-7c646bcfdc-lrp24\" (UID: \"0f093c3c-bac1-4564-91fd-cf382892a875\") " pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:38.313952 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.313865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wtm9\" (UniqueName: \"kubernetes.io/projected/279b5a0c-0445-4e58-b17a-e7d00fb17e18-kube-api-access-5wtm9\") pod \"maas-controller-68c5cd4545-5k6cj\" (UID: \"279b5a0c-0445-4e58-b17a-e7d00fb17e18\") " pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:38.322829 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.322810 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8trj\" (UniqueName: \"kubernetes.io/projected/0f093c3c-bac1-4564-91fd-cf382892a875-kube-api-access-x8trj\") pod \"maas-controller-7c646bcfdc-lrp24\" (UID: \"0f093c3c-bac1-4564-91fd-cf382892a875\") " pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:38.414875 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.414819 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5wtm9\" (UniqueName: \"kubernetes.io/projected/279b5a0c-0445-4e58-b17a-e7d00fb17e18-kube-api-access-5wtm9\") pod \"maas-controller-68c5cd4545-5k6cj\" (UID: \"279b5a0c-0445-4e58-b17a-e7d00fb17e18\") " pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:38.423331 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.423312 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wtm9\" (UniqueName: \"kubernetes.io/projected/279b5a0c-0445-4e58-b17a-e7d00fb17e18-kube-api-access-5wtm9\") pod \"maas-controller-68c5cd4545-5k6cj\" (UID: \"279b5a0c-0445-4e58-b17a-e7d00fb17e18\") " pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:38.423429 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.423380 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:38.591768 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.591738 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:38.598130 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.598100 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c646bcfdc-lrp24"] Jun 03 16:57:38.600135 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:57:38.600103 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f093c3c_bac1_4564_91fd_cf382892a875.slice/crio-61e1a1f91433e31dce1e357e6e9b82f6d6877816804ebaa3a5dec752f93fb64a WatchSource:0}: Error finding container 61e1a1f91433e31dce1e357e6e9b82f6d6877816804ebaa3a5dec752f93fb64a: Status 404 returned error can't find the container with id 61e1a1f91433e31dce1e357e6e9b82f6d6877816804ebaa3a5dec752f93fb64a Jun 03 16:57:38.745765 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:38.745740 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c5cd4545-5k6cj"] Jun 03 16:57:38.747279 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:57:38.747241 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279b5a0c_0445_4e58_b17a_e7d00fb17e18.slice/crio-db14b8044b897ec09a7dc21482b1cb9210251a518e8ed4c0c3f8eecb2668e537 WatchSource:0}: Error finding container db14b8044b897ec09a7dc21482b1cb9210251a518e8ed4c0c3f8eecb2668e537: Status 404 returned error can't find the container with id db14b8044b897ec09a7dc21482b1cb9210251a518e8ed4c0c3f8eecb2668e537 Jun 03 16:57:39.015282 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:39.015210 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" event={"ID":"279b5a0c-0445-4e58-b17a-e7d00fb17e18","Type":"ContainerStarted","Data":"db14b8044b897ec09a7dc21482b1cb9210251a518e8ed4c0c3f8eecb2668e537"} Jun 03 16:57:39.016382 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:39.016346 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" event={"ID":"0f093c3c-bac1-4564-91fd-cf382892a875","Type":"ContainerStarted","Data":"61e1a1f91433e31dce1e357e6e9b82f6d6877816804ebaa3a5dec752f93fb64a"} Jun 03 16:57:42.035367 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:42.035326 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" event={"ID":"0f093c3c-bac1-4564-91fd-cf382892a875","Type":"ContainerStarted","Data":"19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320"} Jun 03 16:57:42.035811 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:42.035400 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:42.036662 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:42.036637 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" event={"ID":"279b5a0c-0445-4e58-b17a-e7d00fb17e18","Type":"ContainerStarted","Data":"fc8ea82c02b9929d64fbb3077635b18bc79a35b2caa7bc47cbdce98c9a7b97a4"} Jun 03 16:57:42.036766 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:42.036722 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:42.053826 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:42.053777 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" podStartSLOduration=1.461985474 podStartE2EDuration="4.053762071s" podCreationTimestamp="2026-06-03 16:57:38 +0000 UTC" firstStartedPulling="2026-06-03 16:57:38.601735776 +0000 UTC m=+830.985083173" lastFinishedPulling="2026-06-03 16:57:41.19351236 +0000 UTC m=+833.576859770" observedRunningTime="2026-06-03 16:57:42.04972515 +0000 UTC m=+834.433072569" watchObservedRunningTime="2026-06-03 16:57:42.053762071 +0000 UTC m=+834.437109491" Jun 03 16:57:42.070108 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:42.070068 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" podStartSLOduration=1.614533378 podStartE2EDuration="4.070056984s" podCreationTimestamp="2026-06-03 16:57:38 +0000 UTC" firstStartedPulling="2026-06-03 16:57:38.748671383 +0000 UTC m=+831.132018780" lastFinishedPulling="2026-06-03 16:57:41.204194978 +0000 UTC m=+833.587542386" observedRunningTime="2026-06-03 16:57:42.068552803 +0000 UTC m=+834.451900224" watchObservedRunningTime="2026-06-03 16:57:42.070056984 +0000 UTC m=+834.453404402" Jun 03 16:57:53.046445 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.046412 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68c5cd4545-5k6cj" Jun 03 16:57:53.046960 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.046759 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:53.105895 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.105857 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c646bcfdc-lrp24"] Jun 03 16:57:53.106123 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.106080 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" podUID="0f093c3c-bac1-4564-91fd-cf382892a875" containerName="manager" containerID="cri-o://19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320" gracePeriod=30 Jun 03 16:57:53.348626 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.348603 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:53.438648 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.438621 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8trj\" (UniqueName: \"kubernetes.io/projected/0f093c3c-bac1-4564-91fd-cf382892a875-kube-api-access-x8trj\") pod \"0f093c3c-bac1-4564-91fd-cf382892a875\" (UID: \"0f093c3c-bac1-4564-91fd-cf382892a875\") " Jun 03 16:57:53.440761 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.440733 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f093c3c-bac1-4564-91fd-cf382892a875-kube-api-access-x8trj" (OuterVolumeSpecName: "kube-api-access-x8trj") pod "0f093c3c-bac1-4564-91fd-cf382892a875" (UID: "0f093c3c-bac1-4564-91fd-cf382892a875"). InnerVolumeSpecName "kube-api-access-x8trj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:57:53.539452 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:53.539426 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8trj\" (UniqueName: \"kubernetes.io/projected/0f093c3c-bac1-4564-91fd-cf382892a875-kube-api-access-x8trj\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:57:54.081776 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.081744 2578 generic.go:358] "Generic (PLEG): container finished" podID="0f093c3c-bac1-4564-91fd-cf382892a875" containerID="19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320" exitCode=0 Jun 03 16:57:54.082212 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.081782 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" event={"ID":"0f093c3c-bac1-4564-91fd-cf382892a875","Type":"ContainerDied","Data":"19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320"} Jun 03 16:57:54.082212 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.081801 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" Jun 03 16:57:54.082212 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.081818 2578 scope.go:117] "RemoveContainer" containerID="19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320" Jun 03 16:57:54.082212 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.081806 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c646bcfdc-lrp24" event={"ID":"0f093c3c-bac1-4564-91fd-cf382892a875","Type":"ContainerDied","Data":"61e1a1f91433e31dce1e357e6e9b82f6d6877816804ebaa3a5dec752f93fb64a"} Jun 03 16:57:54.091081 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.091062 2578 scope.go:117] "RemoveContainer" containerID="19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320" Jun 03 16:57:54.091348 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:57:54.091327 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320\": container with ID starting with 19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320 not found: ID does not exist" containerID="19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320" Jun 03 16:57:54.091406 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.091355 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320"} err="failed to get container status \"19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320\": rpc error: code = NotFound desc = could not find container \"19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320\": container with ID starting with 19f6591d9aa81f6a486dbd704a1788ee8facf5d6c48aace2cb00a57dd805f320 not found: ID does not exist" Jun 03 16:57:54.101853 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.101830 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c646bcfdc-lrp24"] Jun 03 16:57:54.105545 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:54.105525 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c646bcfdc-lrp24"] Jun 03 16:57:56.105675 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:57:56.105639 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f093c3c-bac1-4564-91fd-cf382892a875" path="/var/lib/kubelet/pods/0f093c3c-bac1-4564-91fd-cf382892a875/volumes" Jun 03 16:58:11.484451 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.484407 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-566b79f449-kc7r9"] Jun 03 16:58:11.484934 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.484866 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f093c3c-bac1-4564-91fd-cf382892a875" containerName="manager" Jun 03 16:58:11.484934 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.484886 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f093c3c-bac1-4564-91fd-cf382892a875" containerName="manager" Jun 03 16:58:11.485011 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.484956 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="0f093c3c-bac1-4564-91fd-cf382892a875" containerName="manager" Jun 03 16:58:11.488062 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.488047 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.490242 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.490222 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 03 16:58:11.490508 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.490491 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-gd45d\"" Jun 03 16:58:11.500761 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.500741 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-566b79f449-kc7r9"] Jun 03 16:58:11.579716 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.579687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3ef3aa-b641-449d-9934-b7660ca52f94-maas-api-tls\") pod \"maas-api-566b79f449-kc7r9\" (UID: \"ec3ef3aa-b641-449d-9934-b7660ca52f94\") " pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.579847 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.579720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64w5c\" (UniqueName: \"kubernetes.io/projected/ec3ef3aa-b641-449d-9934-b7660ca52f94-kube-api-access-64w5c\") pod \"maas-api-566b79f449-kc7r9\" (UID: \"ec3ef3aa-b641-449d-9934-b7660ca52f94\") " pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.680519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.680485 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3ef3aa-b641-449d-9934-b7660ca52f94-maas-api-tls\") pod \"maas-api-566b79f449-kc7r9\" (UID: \"ec3ef3aa-b641-449d-9934-b7660ca52f94\") " pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.680519 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.680524 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-64w5c\" (UniqueName: \"kubernetes.io/projected/ec3ef3aa-b641-449d-9934-b7660ca52f94-kube-api-access-64w5c\") pod \"maas-api-566b79f449-kc7r9\" (UID: \"ec3ef3aa-b641-449d-9934-b7660ca52f94\") " pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.683042 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.683017 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3ef3aa-b641-449d-9934-b7660ca52f94-maas-api-tls\") pod \"maas-api-566b79f449-kc7r9\" (UID: \"ec3ef3aa-b641-449d-9934-b7660ca52f94\") " pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.696187 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.696162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64w5c\" (UniqueName: \"kubernetes.io/projected/ec3ef3aa-b641-449d-9934-b7660ca52f94-kube-api-access-64w5c\") pod \"maas-api-566b79f449-kc7r9\" (UID: \"ec3ef3aa-b641-449d-9934-b7660ca52f94\") " pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.798257 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.798167 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:11.908040 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.908011 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-j4vm8"] Jun 03 16:58:11.912848 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.912826 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:11.914843 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.914822 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 03 16:58:11.914968 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.914838 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fqb6d\"" Jun 03 16:58:11.920116 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.920085 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-j4vm8"] Jun 03 16:58:11.946003 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.945977 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-566b79f449-kc7r9"] Jun 03 16:58:11.949938 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:58:11.949908 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3ef3aa_b641_449d_9934_b7660ca52f94.slice/crio-c6412b93f6681897927931f96d7916b0a1539a96ff2cf8601f985f7b4adb48a8 WatchSource:0}: Error finding container c6412b93f6681897927931f96d7916b0a1539a96ff2cf8601f985f7b4adb48a8: Status 404 returned error can't find the container with id c6412b93f6681897927931f96d7916b0a1539a96ff2cf8601f985f7b4adb48a8 Jun 03 16:58:11.983413 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:11.983382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85rcs\" (UniqueName: \"kubernetes.io/projected/d39b7878-cb5c-428e-af25-d3ab4b98ab91-kube-api-access-85rcs\") pod \"payload-processing-58b6c8fdc7-j4vm8\" (UID: \"d39b7878-cb5c-428e-af25-d3ab4b98ab91\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:12.084361 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:12.084265 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-85rcs\" (UniqueName: \"kubernetes.io/projected/d39b7878-cb5c-428e-af25-d3ab4b98ab91-kube-api-access-85rcs\") pod \"payload-processing-58b6c8fdc7-j4vm8\" (UID: \"d39b7878-cb5c-428e-af25-d3ab4b98ab91\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:12.094832 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:12.094810 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85rcs\" (UniqueName: \"kubernetes.io/projected/d39b7878-cb5c-428e-af25-d3ab4b98ab91-kube-api-access-85rcs\") pod \"payload-processing-58b6c8fdc7-j4vm8\" (UID: \"d39b7878-cb5c-428e-af25-d3ab4b98ab91\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:12.155067 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:12.155035 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-566b79f449-kc7r9" event={"ID":"ec3ef3aa-b641-449d-9934-b7660ca52f94","Type":"ContainerStarted","Data":"c6412b93f6681897927931f96d7916b0a1539a96ff2cf8601f985f7b4adb48a8"} Jun 03 16:58:12.225949 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:12.225919 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:12.359065 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:12.359038 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-j4vm8"] Jun 03 16:58:12.360822 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:58:12.360792 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39b7878_cb5c_428e_af25_d3ab4b98ab91.slice/crio-74186cafc8064cddffd985568953b7ddeb3af17d6f8b81ec44e59f719ad1b4f8 WatchSource:0}: Error finding container 74186cafc8064cddffd985568953b7ddeb3af17d6f8b81ec44e59f719ad1b4f8: Status 404 returned error can't find the container with id 74186cafc8064cddffd985568953b7ddeb3af17d6f8b81ec44e59f719ad1b4f8 Jun 03 16:58:13.163387 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:13.163305 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" event={"ID":"d39b7878-cb5c-428e-af25-d3ab4b98ab91","Type":"ContainerStarted","Data":"74186cafc8064cddffd985568953b7ddeb3af17d6f8b81ec44e59f719ad1b4f8"} Jun 03 16:58:15.172230 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:15.172187 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" event={"ID":"d39b7878-cb5c-428e-af25-d3ab4b98ab91","Type":"ContainerStarted","Data":"24ca6bd441846e80cf5c612d33a001387a88fddb26134f9ec619b425d1ebf082"} Jun 03 16:58:15.172699 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:15.172320 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:15.173704 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:15.173678 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-566b79f449-kc7r9" event={"ID":"ec3ef3aa-b641-449d-9934-b7660ca52f94","Type":"ContainerStarted","Data":"9c7c7abfc6c0d8c38595268be38867646d9d3f28ee33a18be4d58b756fdb157a"} Jun 03 16:58:15.173872 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:15.173856 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:15.193424 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:15.193378 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" podStartSLOduration=1.920181388 podStartE2EDuration="4.193367636s" podCreationTimestamp="2026-06-03 16:58:11 +0000 UTC" firstStartedPulling="2026-06-03 16:58:12.36283411 +0000 UTC m=+864.746181521" lastFinishedPulling="2026-06-03 16:58:14.636020363 +0000 UTC m=+867.019367769" observedRunningTime="2026-06-03 16:58:15.189853128 +0000 UTC m=+867.573200558" watchObservedRunningTime="2026-06-03 16:58:15.193367636 +0000 UTC m=+867.576715054" Jun 03 16:58:15.209583 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:15.209520 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-566b79f449-kc7r9" podStartSLOduration=1.573435154 podStartE2EDuration="4.209507287s" podCreationTimestamp="2026-06-03 16:58:11 +0000 UTC" firstStartedPulling="2026-06-03 16:58:11.951634659 +0000 UTC m=+864.334982060" lastFinishedPulling="2026-06-03 16:58:14.587706794 +0000 UTC m=+866.971054193" observedRunningTime="2026-06-03 16:58:15.207413551 +0000 UTC m=+867.590760981" watchObservedRunningTime="2026-06-03 16:58:15.209507287 +0000 UTC m=+867.592854710" Jun 03 16:58:21.182652 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:21.182622 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-566b79f449-kc7r9" Jun 03 16:58:22.910256 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:22.910220 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:58:22.910666 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:22.910462 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" containerID="cri-o://fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8" gracePeriod=30 Jun 03 16:58:24.475655 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:24.475631 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:24.591348 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:24.591322 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnf6f\" (UniqueName: \"kubernetes.io/projected/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3-kube-api-access-dnf6f\") pod \"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3\" (UID: \"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3\") " Jun 03 16:58:24.593334 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:24.593307 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3-kube-api-access-dnf6f" (OuterVolumeSpecName: "kube-api-access-dnf6f") pod "e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" (UID: "e43dd2f4-9fc4-4283-8d20-20be0ae36bf3"). InnerVolumeSpecName "kube-api-access-dnf6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:58:24.692686 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:24.692659 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnf6f\" (UniqueName: \"kubernetes.io/projected/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3-kube-api-access-dnf6f\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:58:25.210702 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.210669 2578 generic.go:358] "Generic (PLEG): container finished" podID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerID="fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8" exitCode=143 Jun 03 16:58:25.210879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.210723 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.210879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.210748 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3","Type":"ContainerDied","Data":"fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8"} Jun 03 16:58:25.210879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.210783 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"e43dd2f4-9fc4-4283-8d20-20be0ae36bf3","Type":"ContainerDied","Data":"899b1b687452e782e2cee92fa343846b49f1b1d819fdafe258809b6a8a2f3e74"} Jun 03 16:58:25.210879 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.210798 2578 scope.go:117] "RemoveContainer" containerID="fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8" Jun 03 16:58:25.221539 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.221521 2578 scope.go:117] "RemoveContainer" containerID="fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8" Jun 03 16:58:25.221794 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:58:25.221778 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8\": container with ID starting with fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8 not found: ID does not exist" containerID="fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8" Jun 03 16:58:25.221844 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.221803 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8"} err="failed to get container status \"fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8\": rpc error: code = NotFound desc = could not find container \"fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8\": container with ID starting with fe8650b7c0e0d36046e1044b0bf69c3f491368906f230837b30c0d15019286a8 not found: ID does not exist" Jun 03 16:58:25.233786 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.233763 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:58:25.238820 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.238797 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:58:25.262706 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.262683 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:58:25.263019 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.263007 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" Jun 03 16:58:25.263064 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.263021 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" Jun 03 16:58:25.263099 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.263081 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" containerName="keycloak" Jun 03 16:58:25.267490 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.267474 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.269703 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.269682 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 03 16:58:25.269703 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.269694 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 03 16:58:25.269875 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.269739 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 03 16:58:25.269875 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.269804 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-mm8wk\"" Jun 03 16:58:25.270026 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.270011 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 03 16:58:25.275122 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.275101 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:58:25.397357 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.397329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/2ef8d55f-c45b-429a-9549-a854277338fa-test-realms\") pod \"maas-keycloak-0\" (UID: \"2ef8d55f-c45b-429a-9549-a854277338fa\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.397499 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.397380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8flpt\" (UniqueName: \"kubernetes.io/projected/2ef8d55f-c45b-429a-9549-a854277338fa-kube-api-access-8flpt\") pod \"maas-keycloak-0\" (UID: \"2ef8d55f-c45b-429a-9549-a854277338fa\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.497928 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.497858 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/2ef8d55f-c45b-429a-9549-a854277338fa-test-realms\") pod \"maas-keycloak-0\" (UID: \"2ef8d55f-c45b-429a-9549-a854277338fa\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.497928 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.497900 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8flpt\" (UniqueName: \"kubernetes.io/projected/2ef8d55f-c45b-429a-9549-a854277338fa-kube-api-access-8flpt\") pod \"maas-keycloak-0\" (UID: \"2ef8d55f-c45b-429a-9549-a854277338fa\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.498659 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.498636 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/2ef8d55f-c45b-429a-9549-a854277338fa-test-realms\") pod \"maas-keycloak-0\" (UID: \"2ef8d55f-c45b-429a-9549-a854277338fa\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.505427 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.505405 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8flpt\" (UniqueName: \"kubernetes.io/projected/2ef8d55f-c45b-429a-9549-a854277338fa-kube-api-access-8flpt\") pod \"maas-keycloak-0\" (UID: \"2ef8d55f-c45b-429a-9549-a854277338fa\") " pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.577654 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.577632 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:25.923550 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:25.923527 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 03 16:58:25.925204 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:58:25.925178 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef8d55f_c45b_429a_9549_a854277338fa.slice/crio-d33840145fd1fde2888141495114f5d8f12e7188492ede93080769dc5e8ea2e4 WatchSource:0}: Error finding container d33840145fd1fde2888141495114f5d8f12e7188492ede93080769dc5e8ea2e4: Status 404 returned error can't find the container with id d33840145fd1fde2888141495114f5d8f12e7188492ede93080769dc5e8ea2e4 Jun 03 16:58:26.100706 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:26.100675 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e43dd2f4-9fc4-4283-8d20-20be0ae36bf3" path="/var/lib/kubelet/pods/e43dd2f4-9fc4-4283-8d20-20be0ae36bf3/volumes" Jun 03 16:58:26.182722 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:26.179961 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-j4vm8" Jun 03 16:58:26.215798 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:26.215764 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"2ef8d55f-c45b-429a-9549-a854277338fa","Type":"ContainerStarted","Data":"d33840145fd1fde2888141495114f5d8f12e7188492ede93080769dc5e8ea2e4"} Jun 03 16:58:27.224105 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:27.224060 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"2ef8d55f-c45b-429a-9549-a854277338fa","Type":"ContainerStarted","Data":"01d6a4946f5f513eb100c69fb1dd1aab8f5aba5bfcc8db5c7be571dcf2865d68"} Jun 03 16:58:27.249309 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:27.249245 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.8422821950000001 podStartE2EDuration="2.249226393s" podCreationTimestamp="2026-06-03 16:58:25 +0000 UTC" firstStartedPulling="2026-06-03 16:58:25.926430546 +0000 UTC m=+878.309777942" lastFinishedPulling="2026-06-03 16:58:26.333374743 +0000 UTC m=+878.716722140" observedRunningTime="2026-06-03 16:58:27.248064576 +0000 UTC m=+879.631411999" watchObservedRunningTime="2026-06-03 16:58:27.249226393 +0000 UTC m=+879.632573814" Jun 03 16:58:27.578813 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:27.578720 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:27.580291 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:27.580256 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:28.578607 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:28.578533 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:29.578550 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:29.578495 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:30.578616 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:30.578186 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:31.579113 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:31.579043 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:32.578665 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:32.578540 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:33.578988 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:33.578936 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:34.578254 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:34.578203 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:35.578078 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:35.578036 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:35.578540 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:35.578452 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:36.578900 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:36.578842 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:37.578570 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:37.578513 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:38.578552 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:38.578493 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 03 16:58:39.686680 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:39.686642 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 03 16:58:39.701413 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:39.701365 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="2ef8d55f-c45b-429a-9549-a854277338fa" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:58:48.079891 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:48.079859 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:58:48.084396 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:48.084370 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 16:58:49.693011 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:58:49.692980 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 03 16:59:00.454577 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.454527 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-98b5ff5b8-c67st"] Jun 03 16:59:00.461774 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.461749 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.463954 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.463930 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 03 16:59:00.466511 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.466491 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-98b5ff5b8-c67st"] Jun 03 16:59:00.528187 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.528159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/69916ba2-43d5-48f9-987e-eb014c798fb1-tls-cert\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.528303 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.528219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqhkf\" (UniqueName: \"kubernetes.io/projected/69916ba2-43d5-48f9-987e-eb014c798fb1-kube-api-access-sqhkf\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.528303 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.528272 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/69916ba2-43d5-48f9-987e-eb014c798fb1-oidc-ca\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.628941 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.628912 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/69916ba2-43d5-48f9-987e-eb014c798fb1-tls-cert\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.629070 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.628961 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sqhkf\" (UniqueName: \"kubernetes.io/projected/69916ba2-43d5-48f9-987e-eb014c798fb1-kube-api-access-sqhkf\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.629070 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.628985 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/69916ba2-43d5-48f9-987e-eb014c798fb1-oidc-ca\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.629550 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.629533 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/69916ba2-43d5-48f9-987e-eb014c798fb1-oidc-ca\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.631283 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.631261 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/69916ba2-43d5-48f9-987e-eb014c798fb1-tls-cert\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.636423 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.636400 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqhkf\" (UniqueName: \"kubernetes.io/projected/69916ba2-43d5-48f9-987e-eb014c798fb1-kube-api-access-sqhkf\") pod \"authorino-98b5ff5b8-c67st\" (UID: \"69916ba2-43d5-48f9-987e-eb014c798fb1\") " pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.771524 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.771459 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-98b5ff5b8-c67st" Jun 03 16:59:00.897233 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:00.897208 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-98b5ff5b8-c67st"] Jun 03 16:59:00.899371 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:59:00.899343 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69916ba2_43d5_48f9_987e_eb014c798fb1.slice/crio-899c8f9d0a656462c55b5cac7541c6aece30699b3317eea43dd8592b4e547165 WatchSource:0}: Error finding container 899c8f9d0a656462c55b5cac7541c6aece30699b3317eea43dd8592b4e547165: Status 404 returned error can't find the container with id 899c8f9d0a656462c55b5cac7541c6aece30699b3317eea43dd8592b4e547165 Jun 03 16:59:01.383415 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.383386 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-98b5ff5b8-c67st" event={"ID":"69916ba2-43d5-48f9-987e-eb014c798fb1","Type":"ContainerStarted","Data":"c983e90d60b741feb32da173f15f77c3297ccfa6402685f841e946282733eef1"} Jun 03 16:59:01.383535 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.383421 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-98b5ff5b8-c67st" event={"ID":"69916ba2-43d5-48f9-987e-eb014c798fb1","Type":"ContainerStarted","Data":"899c8f9d0a656462c55b5cac7541c6aece30699b3317eea43dd8592b4e547165"} Jun 03 16:59:01.402414 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.402367 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-98b5ff5b8-c67st" podStartSLOduration=1.00557907 podStartE2EDuration="1.402352609s" podCreationTimestamp="2026-06-03 16:59:00 +0000 UTC" firstStartedPulling="2026-06-03 16:59:00.900587217 +0000 UTC m=+913.283934614" lastFinishedPulling="2026-06-03 16:59:01.297360745 +0000 UTC m=+913.680708153" observedRunningTime="2026-06-03 16:59:01.399869328 +0000 UTC m=+913.783216748" watchObservedRunningTime="2026-06-03 16:59:01.402352609 +0000 UTC m=+913.785700050" Jun 03 16:59:01.429903 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.429873 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687744bd96-nmfjq"] Jun 03 16:59:01.430149 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.430125 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-687744bd96-nmfjq" podUID="09f7f3d7-ec22-466c-a152-a4453e01a3bd" containerName="authorino" containerID="cri-o://5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8" gracePeriod=30 Jun 03 16:59:01.713460 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.713431 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:59:01.840633 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.840598 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09f7f3d7-ec22-466c-a152-a4453e01a3bd-tls-cert\") pod \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " Jun 03 16:59:01.841113 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.841091 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcl28\" (UniqueName: \"kubernetes.io/projected/09f7f3d7-ec22-466c-a152-a4453e01a3bd-kube-api-access-qcl28\") pod \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\" (UID: \"09f7f3d7-ec22-466c-a152-a4453e01a3bd\") " Jun 03 16:59:01.843965 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.843929 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f7f3d7-ec22-466c-a152-a4453e01a3bd-kube-api-access-qcl28" (OuterVolumeSpecName: "kube-api-access-qcl28") pod "09f7f3d7-ec22-466c-a152-a4453e01a3bd" (UID: "09f7f3d7-ec22-466c-a152-a4453e01a3bd"). InnerVolumeSpecName "kube-api-access-qcl28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:59:01.853890 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.853856 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09f7f3d7-ec22-466c-a152-a4453e01a3bd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "09f7f3d7-ec22-466c-a152-a4453e01a3bd" (UID: "09f7f3d7-ec22-466c-a152-a4453e01a3bd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:59:01.942961 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.942925 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09f7f3d7-ec22-466c-a152-a4453e01a3bd-tls-cert\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:59:01.942961 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:01.942957 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcl28\" (UniqueName: \"kubernetes.io/projected/09f7f3d7-ec22-466c-a152-a4453e01a3bd-kube-api-access-qcl28\") on node \"ip-10-0-136-251.ec2.internal\" DevicePath \"\"" Jun 03 16:59:02.388069 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.387972 2578 generic.go:358] "Generic (PLEG): container finished" podID="09f7f3d7-ec22-466c-a152-a4453e01a3bd" containerID="5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8" exitCode=0 Jun 03 16:59:02.388069 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.388031 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687744bd96-nmfjq" Jun 03 16:59:02.388274 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.388061 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687744bd96-nmfjq" event={"ID":"09f7f3d7-ec22-466c-a152-a4453e01a3bd","Type":"ContainerDied","Data":"5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8"} Jun 03 16:59:02.388274 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.388101 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687744bd96-nmfjq" event={"ID":"09f7f3d7-ec22-466c-a152-a4453e01a3bd","Type":"ContainerDied","Data":"40fcb3cefcd9bad3095f2e9ccfdc696b508e614999c0d4284d7fbbd65f259e45"} Jun 03 16:59:02.388274 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.388118 2578 scope.go:117] "RemoveContainer" containerID="5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8" Jun 03 16:59:02.396771 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.396755 2578 scope.go:117] "RemoveContainer" containerID="5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8" Jun 03 16:59:02.396999 ip-10-0-136-251 kubenswrapper[2578]: E0603 16:59:02.396982 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8\": container with ID starting with 5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8 not found: ID does not exist" containerID="5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8" Jun 03 16:59:02.397053 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.397018 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8"} err="failed to get container status \"5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8\": rpc error: code = NotFound desc = could not find container \"5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8\": container with ID starting with 5ba3f0cfb6488e58c0816ad425e6d049a82dab9ed509bea39f6edfe08dc540f8 not found: ID does not exist" Jun 03 16:59:02.416975 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.416950 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687744bd96-nmfjq"] Jun 03 16:59:02.425894 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:02.425872 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687744bd96-nmfjq"] Jun 03 16:59:04.101116 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:04.101059 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09f7f3d7-ec22-466c-a152-a4453e01a3bd" path="/var/lib/kubelet/pods/09f7f3d7-ec22-466c-a152-a4453e01a3bd/volumes" Jun 03 16:59:39.434910 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.434873 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2"] Jun 03 16:59:39.437350 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.435460 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f7f3d7-ec22-466c-a152-a4453e01a3bd" containerName="authorino" Jun 03 16:59:39.437350 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.435480 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f7f3d7-ec22-466c-a152-a4453e01a3bd" containerName="authorino" Jun 03 16:59:39.437350 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.435608 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="09f7f3d7-ec22-466c-a152-a4453e01a3bd" containerName="authorino" Jun 03 16:59:39.439466 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.439444 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.441442 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.441420 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 03 16:59:39.441544 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.441442 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 03 16:59:39.441869 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.441849 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 03 16:59:39.441962 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.441853 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n6zvg\"" Jun 03 16:59:39.448292 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.448267 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2"] Jun 03 16:59:39.555871 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.555839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.555871 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.555869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4988c497-30d5-4f73-8169-f6c68c004147-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.556089 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.555888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.556089 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.555919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b59pl\" (UniqueName: \"kubernetes.io/projected/4988c497-30d5-4f73-8169-f6c68c004147-kube-api-access-b59pl\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.556089 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.556042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.556260 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.556140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.656996 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.656973 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657127 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657017 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657127 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657060 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657127 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657086 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4988c497-30d5-4f73-8169-f6c68c004147-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657127 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657113 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657330 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657197 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b59pl\" (UniqueName: \"kubernetes.io/projected/4988c497-30d5-4f73-8169-f6c68c004147-kube-api-access-b59pl\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657448 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657448 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657431 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.657625 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.657511 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.659340 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.659319 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4988c497-30d5-4f73-8169-f6c68c004147-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.659526 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.659509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4988c497-30d5-4f73-8169-f6c68c004147-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.666040 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.666018 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b59pl\" (UniqueName: \"kubernetes.io/projected/4988c497-30d5-4f73-8169-f6c68c004147-kube-api-access-b59pl\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2\" (UID: \"4988c497-30d5-4f73-8169-f6c68c004147\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.752108 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.752051 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:39.885205 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:39.885176 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2"] Jun 03 16:59:39.886778 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:59:39.886749 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4988c497_30d5_4f73_8169_f6c68c004147.slice/crio-68cf232a39ae33e7b459d9a58f76835fe8f0eddf5bd19365bc925f9957caccd2 WatchSource:0}: Error finding container 68cf232a39ae33e7b459d9a58f76835fe8f0eddf5bd19365bc925f9957caccd2: Status 404 returned error can't find the container with id 68cf232a39ae33e7b459d9a58f76835fe8f0eddf5bd19365bc925f9957caccd2 Jun 03 16:59:40.542865 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:40.542833 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" event={"ID":"4988c497-30d5-4f73-8169-f6c68c004147","Type":"ContainerStarted","Data":"68cf232a39ae33e7b459d9a58f76835fe8f0eddf5bd19365bc925f9957caccd2"} Jun 03 16:59:43.560599 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:43.560505 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" event={"ID":"4988c497-30d5-4f73-8169-f6c68c004147","Type":"ContainerStarted","Data":"da2ee0b2706692223e2d6d535d398e1690a790ee6c64c79e5088b8f91de1c63a"} Jun 03 16:59:50.073543 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.073491 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7"] Jun 03 16:59:50.081944 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.081861 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.082611 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.082588 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7"] Jun 03 16:59:50.084664 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.084641 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 03 16:59:50.255465 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.255426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.255673 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.255473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbm6s\" (UniqueName: \"kubernetes.io/projected/a403185d-86e5-4b63-b958-bc8dc5edb19f-kube-api-access-mbm6s\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.255673 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.255617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a403185d-86e5-4b63-b958-bc8dc5edb19f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.255789 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.255695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.255789 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.255730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.255789 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.255775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.356862 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.356833 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357031 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.356870 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mbm6s\" (UniqueName: \"kubernetes.io/projected/a403185d-86e5-4b63-b958-bc8dc5edb19f-kube-api-access-mbm6s\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357031 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.356913 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a403185d-86e5-4b63-b958-bc8dc5edb19f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357031 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.356954 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357031 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.356989 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357031 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.357015 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357387 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.357358 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357512 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.357413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.357512 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.357455 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.359197 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.359168 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a403185d-86e5-4b63-b958-bc8dc5edb19f-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.359357 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.359342 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a403185d-86e5-4b63-b958-bc8dc5edb19f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.364985 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.364966 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbm6s\" (UniqueName: \"kubernetes.io/projected/a403185d-86e5-4b63-b958-bc8dc5edb19f-kube-api-access-mbm6s\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7\" (UID: \"a403185d-86e5-4b63-b958-bc8dc5edb19f\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.392515 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.392494 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" Jun 03 16:59:50.553920 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.553885 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7"] Jun 03 16:59:50.563012 ip-10-0-136-251 kubenswrapper[2578]: W0603 16:59:50.562982 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda403185d_86e5_4b63_b958_bc8dc5edb19f.slice/crio-3305e1ebf48c1baccb5f5308fa68144db74e952014adf0669d985685ccc93797 WatchSource:0}: Error finding container 3305e1ebf48c1baccb5f5308fa68144db74e952014adf0669d985685ccc93797: Status 404 returned error can't find the container with id 3305e1ebf48c1baccb5f5308fa68144db74e952014adf0669d985685ccc93797 Jun 03 16:59:50.591118 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:50.591092 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" event={"ID":"a403185d-86e5-4b63-b958-bc8dc5edb19f","Type":"ContainerStarted","Data":"3305e1ebf48c1baccb5f5308fa68144db74e952014adf0669d985685ccc93797"} Jun 03 16:59:51.596009 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:51.595978 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7" event={"ID":"a403185d-86e5-4b63-b958-bc8dc5edb19f","Type":"ContainerStarted","Data":"53c159b246c6b6d8e3eb48d8409cb21f38c1ec24095a0f0ecf99931636469768"} Jun 03 16:59:56.616855 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:56.616775 2578 generic.go:358] "Generic (PLEG): container finished" podID="4988c497-30d5-4f73-8169-f6c68c004147" containerID="da2ee0b2706692223e2d6d535d398e1690a790ee6c64c79e5088b8f91de1c63a" exitCode=0 Jun 03 16:59:56.616855 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:56.616823 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" event={"ID":"4988c497-30d5-4f73-8169-f6c68c004147","Type":"ContainerDied","Data":"da2ee0b2706692223e2d6d535d398e1690a790ee6c64c79e5088b8f91de1c63a"} Jun 03 16:59:58.625521 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:58.625484 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" event={"ID":"4988c497-30d5-4f73-8169-f6c68c004147","Type":"ContainerStarted","Data":"bbdc6b91734e1f8ad7035589fb44aed842465407a75ea99b24e37e352e003eb9"} Jun 03 16:59:58.648276 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:58.648231 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" podStartSLOduration=1.743712047 podStartE2EDuration="19.648218261s" podCreationTimestamp="2026-06-03 16:59:39 +0000 UTC" firstStartedPulling="2026-06-03 16:59:39.888626392 +0000 UTC m=+952.271973789" lastFinishedPulling="2026-06-03 16:59:57.793132592 +0000 UTC m=+970.176480003" observedRunningTime="2026-06-03 16:59:58.644895526 +0000 UTC m=+971.028242946" watchObservedRunningTime="2026-06-03 16:59:58.648218261 +0000 UTC m=+971.031565679" Jun 03 16:59:59.752755 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:59.752719 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:59.752755 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:59.752762 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 16:59:59.765459 ip-10-0-136-251 kubenswrapper[2578]: I0603 16:59:59.765434 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 17:00:00.649478 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:00:00.649446 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2" Jun 03 17:03:48.129080 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:03:48.129051 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 17:03:48.135296 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:03:48.135277 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lp8cj_b54328d7-952f-4cf4-a355-b6a2a697354c/ovn-acl-logging/0.log" Jun 03 17:06:06.644479 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:06.644395 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7_a403185d-86e5-4b63-b958-bc8dc5edb19f/storage-initializer/0.log" Jun 03 17:06:07.679066 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:07.679036 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98b5ff5b8-c67st_69916ba2-43d5-48f9-987e-eb014c798fb1/authorino/0.log" Jun 03 17:06:12.601408 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:12.601373 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-566b79f449-kc7r9_ec3ef3aa-b641-449d-9934-b7660ca52f94/maas-api/0.log" Jun 03 17:06:12.823237 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:12.823200 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68c5cd4545-5k6cj_279b5a0c-0445-4e58-b17a-e7d00fb17e18/manager/0.log" Jun 03 17:06:13.161598 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:13.161550 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99597487-4jph6_eccef2f7-2352-415f-8eda-cee467d6b719/copy-manifests/0.log" Jun 03 17:06:13.168147 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:13.168127 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99597487-4jph6_eccef2f7-2352-415f-8eda-cee467d6b719/manager/0.log" Jun 03 17:06:14.298703 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.298669 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk_f7f4efc1-432d-4a59-bd54-09b98b89decf/util/0.log" Jun 03 17:06:14.306201 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.306173 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk_f7f4efc1-432d-4a59-bd54-09b98b89decf/pull/0.log" Jun 03 17:06:14.312933 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.312911 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk_f7f4efc1-432d-4a59-bd54-09b98b89decf/extract/0.log" Jun 03 17:06:14.424113 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.424090 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds_a683e7eb-2577-4308-a28e-00bbaf70850e/pull/0.log" Jun 03 17:06:14.430941 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.430920 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds_a683e7eb-2577-4308-a28e-00bbaf70850e/extract/0.log" Jun 03 17:06:14.437086 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.437068 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds_a683e7eb-2577-4308-a28e-00bbaf70850e/util/0.log" Jun 03 17:06:14.548649 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.548624 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8_a95d1e32-a7e9-4673-aa9f-f7f056a8faba/pull/0.log" Jun 03 17:06:14.554781 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.554717 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8_a95d1e32-a7e9-4673-aa9f-f7f056a8faba/extract/0.log" Jun 03 17:06:14.562344 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.562324 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8_a95d1e32-a7e9-4673-aa9f-f7f056a8faba/util/0.log" Jun 03 17:06:14.667703 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.667684 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm_ff83b8fb-6fb8-458a-947d-6a808890cf12/util/0.log" Jun 03 17:06:14.672993 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.672968 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm_ff83b8fb-6fb8-458a-947d-6a808890cf12/pull/0.log" Jun 03 17:06:14.677633 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.677615 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm_ff83b8fb-6fb8-458a-947d-6a808890cf12/extract/0.log" Jun 03 17:06:14.793089 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:14.793066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98b5ff5b8-c67st_69916ba2-43d5-48f9-987e-eb014c798fb1/authorino/0.log" Jun 03 17:06:15.373191 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:15.373166 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-9q9jq_c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3/manager/0.log" Jun 03 17:06:16.239267 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:16.239238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8f98dcdbc-xfbkg_7c022254-cdbc-4a0f-8665-ddca57b5ba71/kube-auth-proxy/0.log" Jun 03 17:06:16.456835 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:16.456793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-j4vm8_d39b7878-cb5c-428e-af25-d3ab4b98ab91/payload-processing/0.log" Jun 03 17:06:17.367774 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:17.367737 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2_4988c497-30d5-4f73-8169-f6c68c004147/storage-initializer/0.log" Jun 03 17:06:17.375056 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:17.375021 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-w8xq2_4988c497-30d5-4f73-8169-f6c68c004147/main/0.log" Jun 03 17:06:17.476621 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:17.476579 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-llbj7_a403185d-86e5-4b63-b958-bc8dc5edb19f/storage-initializer/0.log" Jun 03 17:06:24.441990 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:24.441957 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dkxz2_dcbc39e8-0839-44bd-9fa3-c757d3900a12/global-pull-secret-syncer/0.log" Jun 03 17:06:24.517991 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:24.517963 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ht9gl_142582a7-24a0-4b02-bcba-a4554f275c8e/konnectivity-agent/0.log" Jun 03 17:06:24.572122 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:24.572090 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-251.ec2.internal_77d77f19d5202da2dce27948f5d2589d/haproxy/0.log" Jun 03 17:06:28.433587 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.433546 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk_f7f4efc1-432d-4a59-bd54-09b98b89decf/extract/0.log" Jun 03 17:06:28.457528 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.457500 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk_f7f4efc1-432d-4a59-bd54-09b98b89decf/util/0.log" Jun 03 17:06:28.479194 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.479160 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qp7rk_f7f4efc1-432d-4a59-bd54-09b98b89decf/pull/0.log" Jun 03 17:06:28.508158 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.508138 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds_a683e7eb-2577-4308-a28e-00bbaf70850e/extract/0.log" Jun 03 17:06:28.529500 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.529474 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds_a683e7eb-2577-4308-a28e-00bbaf70850e/util/0.log" Jun 03 17:06:28.559881 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.559858 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6vds_a683e7eb-2577-4308-a28e-00bbaf70850e/pull/0.log" Jun 03 17:06:28.587037 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.587010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8_a95d1e32-a7e9-4673-aa9f-f7f056a8faba/extract/0.log" Jun 03 17:06:28.606511 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.606490 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8_a95d1e32-a7e9-4673-aa9f-f7f056a8faba/util/0.log" Jun 03 17:06:28.634619 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.634574 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmnh8_a95d1e32-a7e9-4673-aa9f-f7f056a8faba/pull/0.log" Jun 03 17:06:28.661157 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.661134 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm_ff83b8fb-6fb8-458a-947d-6a808890cf12/extract/0.log" Jun 03 17:06:28.689376 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.689312 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm_ff83b8fb-6fb8-458a-947d-6a808890cf12/util/0.log" Jun 03 17:06:28.750073 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.750050 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h4hcm_ff83b8fb-6fb8-458a-947d-6a808890cf12/pull/0.log" Jun 03 17:06:28.782055 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.782031 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98b5ff5b8-c67st_69916ba2-43d5-48f9-987e-eb014c798fb1/authorino/0.log" Jun 03 17:06:28.972206 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:28.972142 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-9q9jq_c7d6021a-9e0f-4fa9-9ce2-e35bd9d064f3/manager/0.log" Jun 03 17:06:30.740739 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.740707 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-2lgjn_d8bd50dd-7ddd-4a74-ae63-c6114a93388e/cluster-monitoring-operator/0.log" Jun 03 17:06:30.771908 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.766511 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-vfj4g_45ab17aa-36a7-4214-a570-bca699c6ce5b/kube-state-metrics/0.log" Jun 03 17:06:30.787651 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.787626 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-vfj4g_45ab17aa-36a7-4214-a570-bca699c6ce5b/kube-rbac-proxy-main/0.log" Jun 03 17:06:30.809699 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.809674 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-vfj4g_45ab17aa-36a7-4214-a570-bca699c6ce5b/kube-rbac-proxy-self/0.log" Jun 03 17:06:30.843039 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.843011 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86d49fbb7c-5sb9v_1e34a595-64d4-4375-a219-cd674a3406c3/metrics-server/0.log" Jun 03 17:06:30.974121 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.974092 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj64g_91b99905-fbd8-4832-a92e-b6ba2eef61ec/node-exporter/0.log" Jun 03 17:06:30.999023 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:30.998955 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj64g_91b99905-fbd8-4832-a92e-b6ba2eef61ec/kube-rbac-proxy/0.log" Jun 03 17:06:31.037700 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.037676 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj64g_91b99905-fbd8-4832-a92e-b6ba2eef61ec/init-textfile/0.log" Jun 03 17:06:31.555665 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.555623 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b64457c-qrlft_41057247-908a-4105-b6f1-bb08a0de923e/thanos-query/0.log" Jun 03 17:06:31.575828 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.575794 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b64457c-qrlft_41057247-908a-4105-b6f1-bb08a0de923e/kube-rbac-proxy-web/0.log" Jun 03 17:06:31.595963 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.595932 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b64457c-qrlft_41057247-908a-4105-b6f1-bb08a0de923e/kube-rbac-proxy/0.log" Jun 03 17:06:31.617131 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.617107 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b64457c-qrlft_41057247-908a-4105-b6f1-bb08a0de923e/prom-label-proxy/0.log" Jun 03 17:06:31.640487 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.640462 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b64457c-qrlft_41057247-908a-4105-b6f1-bb08a0de923e/kube-rbac-proxy-rules/0.log" Jun 03 17:06:31.662710 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:31.662691 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b64457c-qrlft_41057247-908a-4105-b6f1-bb08a0de923e/kube-rbac-proxy-metrics/0.log" Jun 03 17:06:32.825863 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:32.825838 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-skmzm_7c753777-850b-4a83-9523-aef26d98fda7/networking-console-plugin/0.log" Jun 03 17:06:33.624159 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.624129 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk"] Jun 03 17:06:33.627825 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.627801 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.630647 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.630623 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6jxqt\"/\"openshift-service-ca.crt\"" Jun 03 17:06:33.630897 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.630881 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6jxqt\"/\"kube-root-ca.crt\"" Jun 03 17:06:33.634128 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.634078 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6jxqt\"/\"default-dockercfg-t9nhb\"" Jun 03 17:06:33.637344 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.637314 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk"] Jun 03 17:06:33.669425 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.669403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-proc\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.669546 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.669433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-lib-modules\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.669546 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.669461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrqv\" (UniqueName: \"kubernetes.io/projected/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-kube-api-access-drrqv\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.669546 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.669506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-sys\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.669546 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.669541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-podres\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770520 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770497 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-proc\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770520 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770526 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-lib-modules\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770713 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770550 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-drrqv\" (UniqueName: \"kubernetes.io/projected/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-kube-api-access-drrqv\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770713 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770588 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-sys\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770713 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770634 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-proc\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770713 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770645 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-sys\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770713 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770684 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-lib-modules\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770887 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770708 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-podres\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.770887 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.770793 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-podres\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.778690 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.778667 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrqv\" (UniqueName: \"kubernetes.io/projected/ac26d8d2-f1ee-4b64-aeec-17bd070a8aac-kube-api-access-drrqv\") pod \"perf-node-gather-daemonset-ck5lk\" (UID: \"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac\") " pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:33.938882 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:33.938802 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:34.003125 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.003084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8687b5b458-kcxfj_5de392ed-47a2-4afb-ba88-3147c2469b08/console/0.log" Jun 03 17:06:34.087311 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.087287 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk"] Jun 03 17:06:34.089280 ip-10-0-136-251 kubenswrapper[2578]: W0603 17:06:34.089253 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podac26d8d2_f1ee_4b64_aeec_17bd070a8aac.slice/crio-1ecdf23bd905bf2a08edf270e3f1407346acb592189a80d341076468af2ace26 WatchSource:0}: Error finding container 1ecdf23bd905bf2a08edf270e3f1407346acb592189a80d341076468af2ace26: Status 404 returned error can't find the container with id 1ecdf23bd905bf2a08edf270e3f1407346acb592189a80d341076468af2ace26 Jun 03 17:06:34.090847 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.090832 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 17:06:34.195676 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.195622 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" event={"ID":"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac","Type":"ContainerStarted","Data":"e8b062dcce5bba9acc3ecb56a0fe4a9bb97cf7eef626a9db3430678b264b1630"} Jun 03 17:06:34.195676 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.195654 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" event={"ID":"ac26d8d2-f1ee-4b64-aeec-17bd070a8aac","Type":"ContainerStarted","Data":"1ecdf23bd905bf2a08edf270e3f1407346acb592189a80d341076468af2ace26"} Jun 03 17:06:34.195806 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.195702 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:34.216780 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:34.216744 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" podStartSLOduration=1.216730508 podStartE2EDuration="1.216730508s" podCreationTimestamp="2026-06-03 17:06:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 17:06:34.213540647 +0000 UTC m=+1366.596888065" watchObservedRunningTime="2026-06-03 17:06:34.216730508 +0000 UTC m=+1366.600077951" Jun 03 17:06:35.501149 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:35.501119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x82rl_ba615b97-7ad1-40fe-868e-e6ec99612afe/dns/0.log" Jun 03 17:06:35.521482 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:35.521456 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x82rl_ba615b97-7ad1-40fe-868e-e6ec99612afe/kube-rbac-proxy/0.log" Jun 03 17:06:35.587382 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:35.587360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmbrr_fa1a1e82-367d-49e7-9abb-6f49ef988e5d/dns-node-resolver/0.log" Jun 03 17:06:36.086896 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:36.086864 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-869d9fc789-8rpgv_eaa859a3-7adb-407e-8c0c-8c8bc4a10070/registry/0.log" Jun 03 17:06:36.094840 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:36.094810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-869d9fc789-8rpgv_eaa859a3-7adb-407e-8c0c-8c8bc4a10070/registry/1.log" Jun 03 17:06:36.178728 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:36.178701 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l74wk_06cc3bd3-c291-4ad8-8515-bb30440a138f/node-ca/0.log" Jun 03 17:06:37.169174 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:37.169138 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8f98dcdbc-xfbkg_7c022254-cdbc-4a0f-8665-ddca57b5ba71/kube-auth-proxy/0.log" Jun 03 17:06:37.218065 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:37.218037 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-j4vm8_d39b7878-cb5c-428e-af25-d3ab4b98ab91/payload-processing/0.log" Jun 03 17:06:37.812802 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:37.812771 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tch6r_a904f235-adff-4af5-b66c-fee8a13025d5/serve-healthcheck-canary/0.log" Jun 03 17:06:38.317447 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:38.317413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-5n57j_71f052c3-b28b-4eff-a502-518f583b7049/insights-operator/0.log" Jun 03 17:06:38.317898 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:38.317670 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-5n57j_71f052c3-b28b-4eff-a502-518f583b7049/insights-operator/1.log" Jun 03 17:06:38.518744 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:38.518717 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cqnsv_db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53/kube-rbac-proxy/0.log" Jun 03 17:06:38.584800 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:38.584775 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cqnsv_db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53/exporter/0.log" Jun 03 17:06:38.627933 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:38.627902 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cqnsv_db3d34e3-09e2-47d7-ae3d-50f9a2a1ee53/extractor/0.log" Jun 03 17:06:40.212706 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:40.212679 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6jxqt/perf-node-gather-daemonset-ck5lk" Jun 03 17:06:40.812670 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:40.812640 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-566b79f449-kc7r9_ec3ef3aa-b641-449d-9934-b7660ca52f94/maas-api/0.log" Jun 03 17:06:40.867192 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:40.867160 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68c5cd4545-5k6cj_279b5a0c-0445-4e58-b17a-e7d00fb17e18/manager/0.log" Jun 03 17:06:40.955328 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:40.955296 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99597487-4jph6_eccef2f7-2352-415f-8eda-cee467d6b719/manager/0.log" Jun 03 17:06:40.978286 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:40.978268 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99597487-4jph6_eccef2f7-2352-415f-8eda-cee467d6b719/copy-manifests/0.log" Jun 03 17:06:42.644412 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:42.644370 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-849d86d84c-522pl_3394bb26-7eda-4c45-912c-d859d3b1859a/manager/0.log" Jun 03 17:06:48.370026 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:48.369995 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-th5r7_604f5077-f50a-4f66-8951-8ff2799b3e82/kube-storage-version-migrator-operator/1.log" Jun 03 17:06:48.371806 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:48.371774 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-th5r7_604f5077-f50a-4f66-8951-8ff2799b3e82/kube-storage-version-migrator-operator/0.log" Jun 03 17:06:49.496521 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.496487 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4nbf5_feea66f6-c2cf-43e6-ac4e-6dd8baeec6e4/kube-multus/0.log" Jun 03 17:06:49.518966 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.518947 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/kube-multus-additional-cni-plugins/0.log" Jun 03 17:06:49.538734 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.538662 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/egress-router-binary-copy/0.log" Jun 03 17:06:49.561433 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.561399 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/cni-plugins/0.log" Jun 03 17:06:49.583745 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.583725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/bond-cni-plugin/0.log" Jun 03 17:06:49.604819 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.604799 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/routeoverride-cni/0.log" Jun 03 17:06:49.626078 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.626058 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/whereabouts-cni-bincopy/0.log" Jun 03 17:06:49.649132 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:49.649110 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8qqp5_4f5d7b02-bc02-4e14-8aa3-5b489f48826c/whereabouts-cni/0.log" Jun 03 17:06:50.148887 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:50.148855 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rn64j_e0cf9100-e9b8-42a0-93a8-f273610a87cf/network-metrics-daemon/0.log" Jun 03 17:06:50.168258 ip-10-0-136-251 kubenswrapper[2578]: I0603 17:06:50.168234 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rn64j_e0cf9100-e9b8-42a0-93a8-f273610a87cf/kube-rbac-proxy/0.log"