Jun 03 01:37:58.314929 ip-10-0-136-201 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 03 01:37:58.314938 ip-10-0-136-201 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 03 01:37:58.314946 ip-10-0-136-201 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 03 01:37:58.315193 ip-10-0-136-201 systemd[1]: Failed to start Kubernetes Kubelet. Jun 03 01:38:08.535307 ip-10-0-136-201 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 03 01:38:08.535327 ip-10-0-136-201 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d7381644b803453892ae5eca46e308eb -- Jun 03 01:40:29.229872 ip-10-0-136-201 systemd[1]: Starting Kubernetes Kubelet... Jun 03 01:40:29.646318 ip-10-0-136-201 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 01:40:29.646318 ip-10-0-136-201 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 01:40:29.646318 ip-10-0-136-201 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 01:40:29.646318 ip-10-0-136-201 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 01:40:29.646318 ip-10-0-136-201 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 01:40:29.647979 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.647807 2553 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 01:40:29.652852 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652836 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 01:40:29.652852 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652852 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652856 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652861 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652864 2553 feature_gate.go:328] unrecognized feature gate: Example Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652867 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652869 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652873 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652876 2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652879 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652881 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652884 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652891 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652894 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652896 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652899 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652901 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652904 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652907 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652909 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652912 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 01:40:29.652926 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652914 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652916 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652919 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652921 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652924 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652926 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652929 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652932 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652934 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652937 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652940 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652943 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652945 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652948 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652950 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652952 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652955 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652957 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652960 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 01:40:29.653398 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652962 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652965 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652967 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652969 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652972 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652977 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652980 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652983 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652985 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652988 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652990 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652992 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652995 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.652997 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653000 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653003 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653006 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653009 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653011 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 01:40:29.653871 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653014 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653018 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653022 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653025 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653028 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653031 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653034 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653038 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653041 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653043 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653046 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653049 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653051 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653054 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653056 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653059 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653062 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653064 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653066 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653069 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 01:40:29.654328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653072 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 01:40:29.654832 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653074 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 01:40:29.654832 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653077 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 01:40:29.654832 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653079 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 01:40:29.654832 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653082 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 01:40:29.654832 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653084 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 01:40:29.654832 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.653087 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 01:40:29.655948 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655935 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 01:40:29.655948 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655947 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655951 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655953 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655956 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655959 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655961 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655964 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655966 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655969 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655971 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655974 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655976 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655979 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655981 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655984 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655986 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655989 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655991 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655994 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655996 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 01:40:29.656008 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.655999 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656001 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656004 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656007 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656010 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656013 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656015 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656018 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656021 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656023 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656026 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656028 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656031 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656034 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656036 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656039 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656041 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656043 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656046 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656048 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 01:40:29.656615 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656050 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656053 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656056 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656058 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656060 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656064 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656068 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656070 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656073 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656076 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656079 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656082 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656085 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656087 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656091 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656094 2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656097 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656100 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656103 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 01:40:29.657405 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656105 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656108 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656110 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656113 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656116 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656118 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656121 2553 feature_gate.go:328] unrecognized feature gate: Example Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656123 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656126 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656129 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656133 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656135 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656234 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656239 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656243 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656246 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656249 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656251 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656254 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656256 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 01:40:29.657944 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656259 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656261 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656265 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656267 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656273 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.656276 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.656632 2553 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.656701 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657453 2553 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657462 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657469 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657515 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657674 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657683 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657688 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657693 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657698 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657703 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657708 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657712 2553 flags.go:64] FLAG: --cgroup-root="" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657716 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657721 2553 flags.go:64] FLAG: --client-ca-file="" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657725 2553 flags.go:64] FLAG: --cloud-config="" Jun 03 01:40:29.658430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657729 2553 flags.go:64] FLAG: --cloud-provider="external" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657734 2553 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657752 2553 flags.go:64] FLAG: --cluster-domain="" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657756 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657761 2553 flags.go:64] FLAG: --config-dir="" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657765 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657770 2553 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657777 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657781 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657786 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657791 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657796 2553 flags.go:64] FLAG: --contention-profiling="false" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657801 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657806 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657811 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657815 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657822 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657827 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657831 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657835 2553 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657852 2553 flags.go:64] FLAG: --enable-server="true" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657860 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657874 2553 flags.go:64] FLAG: --event-burst="100" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657878 2553 flags.go:64] FLAG: --event-qps="50" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657881 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 01:40:29.659040 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657884 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657887 2553 flags.go:64] FLAG: --eviction-hard="" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657894 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657897 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657900 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657904 2553 flags.go:64] FLAG: --eviction-soft="" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657907 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657910 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657912 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657915 2553 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657918 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657921 2553 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657924 2553 flags.go:64] FLAG: --feature-gates="" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657928 2553 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657931 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657934 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657937 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657941 2553 flags.go:64] FLAG: --healthz-port="10248" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657944 2553 flags.go:64] FLAG: --help="false" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657947 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-136-201.ec2.internal" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657951 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657954 2553 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657957 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657961 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 01:40:29.659684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657964 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657967 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657970 2553 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657974 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657977 2553 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657980 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657983 2553 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657985 2553 flags.go:64] FLAG: --kube-reserved="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657988 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657991 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657994 2553 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.657997 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658000 2553 flags.go:64] FLAG: --lock-file="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658002 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658006 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658009 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658014 2553 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658017 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658020 2553 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658022 2553 flags.go:64] FLAG: --logging-format="text" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658025 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658029 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658032 2553 flags.go:64] FLAG: --manifest-url="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658034 2553 flags.go:64] FLAG: --manifest-url-header="" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658039 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 01:40:29.660274 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658042 2553 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658047 2553 flags.go:64] FLAG: --max-pods="110" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658050 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658053 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658055 2553 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658058 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658061 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658064 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658067 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658075 2553 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658078 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658081 2553 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658084 2553 flags.go:64] FLAG: --pod-cidr="" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658087 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658093 2553 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658096 2553 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658099 2553 flags.go:64] FLAG: --pods-per-core="0" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658102 2553 flags.go:64] FLAG: --port="10250" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658104 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658107 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05ea1f9bd19c10b32" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658110 2553 flags.go:64] FLAG: --qos-reserved="" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658113 2553 flags.go:64] FLAG: --read-only-port="10255" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658116 2553 flags.go:64] FLAG: --register-node="true" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658119 2553 flags.go:64] FLAG: --register-schedulable="true" Jun 03 01:40:29.660951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658122 2553 flags.go:64] FLAG: --register-with-taints="" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658125 2553 flags.go:64] FLAG: --registry-burst="10" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658128 2553 flags.go:64] FLAG: --registry-qps="5" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658131 2553 flags.go:64] FLAG: --reserved-cpus="" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658134 2553 flags.go:64] FLAG: --reserved-memory="" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658137 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658140 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658143 2553 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658145 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658148 2553 flags.go:64] FLAG: --runonce="false" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658154 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658157 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658160 2553 flags.go:64] FLAG: --seccomp-default="false" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658163 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658166 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658170 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658173 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658176 2553 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658179 2553 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658182 2553 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658185 2553 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658188 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658190 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658193 2553 flags.go:64] FLAG: --system-cgroups="" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658196 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 01:40:29.661561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658202 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658205 2553 flags.go:64] FLAG: --tls-cert-file="" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658208 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658212 2553 flags.go:64] FLAG: --tls-min-version="" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658215 2553 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658217 2553 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658221 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658223 2553 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658226 2553 flags.go:64] FLAG: --v="2" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658235 2553 flags.go:64] FLAG: --version="false" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658239 2553 flags.go:64] FLAG: --vmodule="" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658243 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.658247 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658352 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658355 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658358 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658361 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658368 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658371 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658373 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658376 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658380 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 01:40:29.662179 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658384 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658387 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658389 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658392 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658395 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658398 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658400 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658403 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658406 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658408 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658410 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658413 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658415 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658418 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658420 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658423 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658425 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658427 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658430 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658433 2553 feature_gate.go:328] unrecognized feature gate: Example Jun 03 01:40:29.662758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658435 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658438 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658440 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658443 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658445 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658448 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658450 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658453 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658455 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658458 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658460 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658462 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658465 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658467 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658470 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658473 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658476 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658478 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658481 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658483 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 01:40:29.663264 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658486 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658488 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658490 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658492 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658497 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658500 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658503 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658505 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658508 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658511 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658527 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658545 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658549 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658551 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658554 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658557 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658559 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658562 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658564 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 01:40:29.663788 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658567 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658570 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658572 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658574 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658577 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658579 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658582 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658585 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658588 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658591 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658594 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658597 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658599 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658602 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658604 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658606 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658609 2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 01:40:29.664250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.658611 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 01:40:29.664738 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.659266 2553 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 01:40:29.666153 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.666134 2553 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 01:40:29.666194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.666154 2553 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666200 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666205 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666209 2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666212 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666215 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666217 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666221 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666224 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666226 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 01:40:29.666226 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666229 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666233 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666235 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666238 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666240 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666243 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666246 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666248 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666251 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666253 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666255 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666258 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666260 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666262 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666265 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666267 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666270 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666272 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666274 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666277 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 01:40:29.666478 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666279 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666282 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666285 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666288 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666291 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666293 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666296 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666298 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666301 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666303 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666306 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666308 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666311 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666313 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666316 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666319 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666321 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666324 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666326 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666328 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 01:40:29.666990 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666331 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666333 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666336 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666338 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666340 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666343 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666345 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666347 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666350 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666352 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666355 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666358 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666360 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666362 2553 feature_gate.go:328] unrecognized feature gate: Example Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666365 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666368 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666370 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666373 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666377 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 01:40:29.667471 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666381 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666384 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666387 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666391 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666395 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666398 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666401 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666404 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666407 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666409 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666412 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666415 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666418 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666420 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666423 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666426 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666429 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 01:40:29.667957 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666431 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.666437 2553 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666564 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666571 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666573 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666576 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666579 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666582 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666584 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666587 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666590 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666592 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666595 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666598 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666601 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666603 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 01:40:29.668376 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666606 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666608 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666611 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666613 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666616 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666618 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666620 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666623 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666625 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666628 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666630 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666632 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666635 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666637 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666640 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666642 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666644 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666647 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666650 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 01:40:29.668786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666654 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666656 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666658 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666662 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666665 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666668 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666671 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666674 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666676 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666679 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666683 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666686 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666689 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666691 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666694 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666697 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666699 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666702 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666704 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 01:40:29.669240 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666707 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666709 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666729 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666733 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666736 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666739 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666742 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666745 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666748 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666750 2553 feature_gate.go:328] unrecognized feature gate: Example Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666753 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666755 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666758 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666761 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666763 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666766 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666769 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666772 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666775 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666778 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666780 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 01:40:29.669800 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666783 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666786 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666788 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666791 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666794 2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666796 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666799 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666801 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666804 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666806 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666808 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666811 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:29.666813 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.666818 2553 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.666946 2553 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 01:40:29.670352 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.669764 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 01:40:29.671083 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.671071 2553 server.go:1019] "Starting client certificate rotation" Jun 03 01:40:29.671197 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.671182 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 01:40:29.671235 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.671221 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 01:40:29.694224 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.694201 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 01:40:29.695878 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.695856 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 01:40:29.711377 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.711345 2553 log.go:25] "Validated CRI v1 runtime API" Jun 03 01:40:29.721557 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.721516 2553 log.go:25] "Validated CRI v1 image API" Jun 03 01:40:29.723887 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.723856 2553 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 01:40:29.726952 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.726928 2553 fs.go:135] Filesystem UUIDs: map[34e0ad66-506a-40e5-af4e-13348eff1859:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9d05a5ef-ca89-4cdf-86ac-401aa49daf27:/dev/nvme0n1p4] Jun 03 01:40:29.727026 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.726953 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 01:40:29.729346 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.729327 2553 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 01:40:29.731640 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.731512 2553 manager.go:217] Machine: {Timestamp:2026-06-03 01:40:29.730987274 +0000 UTC m=+0.385630545 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100467 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27be8a7d349a24eb8314fc59c9b043 SystemUUID:ec27be8a-7d34-9a24-eb83-14fc59c9b043 BootID:d7381644-b803-4538-92ae-5eca46e308eb Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:67:53:89:42:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:67:53:89:42:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:ab:c8:4d:57:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 01:40:29.732299 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.732287 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 01:40:29.732393 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.732381 2553 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 01:40:29.735781 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.735751 2553 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 01:40:29.735927 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.735784 2553 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-201.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 01:40:29.735975 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.735937 2553 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 01:40:29.735975 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.735947 2553 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 01:40:29.735975 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.735954 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 01:40:29.736670 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.736660 2553 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 01:40:29.738577 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.738565 2553 state_mem.go:36] "Initialized new in-memory state store" Jun 03 01:40:29.738702 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.738694 2553 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 01:40:29.740985 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.740975 2553 kubelet.go:491] "Attempting to sync node with API server" Jun 03 01:40:29.741026 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.740990 2553 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 01:40:29.741026 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.741003 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 01:40:29.741026 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.741012 2553 kubelet.go:397] "Adding apiserver pod source" Jun 03 01:40:29.741026 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.741020 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 01:40:29.742035 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.742021 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 01:40:29.742072 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.742043 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 01:40:29.744914 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.744899 2553 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 01:40:29.748517 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.748500 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lsf5h" Jun 03 01:40:29.753826 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.753811 2553 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 01:40:29.755763 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.755741 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lsf5h" Jun 03 01:40:29.756295 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.756275 2553 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756305 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756326 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756333 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756339 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756347 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756353 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 01:40:29.756360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756358 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 01:40:29.756552 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756365 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 01:40:29.756552 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756376 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 01:40:29.756552 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756383 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 01:40:29.756552 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756397 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 01:40:29.756552 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.756403 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 01:40:29.757090 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.757070 2553 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 01:40:29.757945 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.757934 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 01:40:29.757945 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.757945 2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 01:40:29.761174 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.761159 2553 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 01:40:29.761235 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.761214 2553 server.go:1295] "Started kubelet" Jun 03 01:40:29.761376 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.761303 2553 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 01:40:29.761451 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.761437 2553 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 01:40:29.761564 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.761333 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 01:40:29.762317 ip-10-0-136-201 systemd[1]: Started Kubernetes Kubelet. Jun 03 01:40:29.762882 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.762625 2553 server.go:317] "Adding debug handlers to kubelet server" Jun 03 01:40:29.764848 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.764830 2553 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 01:40:29.771190 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.771172 2553 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 01:40:29.772077 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.772060 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 01:40:29.772643 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.772628 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 01:40:29.772803 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.772779 2553 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 01:40:29.772803 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.772795 2553 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 01:40:29.772941 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.772840 2553 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 01:40:29.776735 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.776714 2553 reconstruct.go:97] "Volume reconstruction finished" Jun 03 01:40:29.776735 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.776733 2553 reconciler.go:26] "Reconciler: start to sync state" Jun 03 01:40:29.777435 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.777387 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:29.777606 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.777584 2553 factory.go:55] Registering systemd factory Jun 03 01:40:29.777712 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.777651 2553 factory.go:223] Registration of the systemd container factory successfully Jun 03 01:40:29.778086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.778068 2553 factory.go:153] Registering CRI-O factory Jun 03 01:40:29.778086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.778086 2553 factory.go:223] Registration of the crio container factory successfully Jun 03 01:40:29.778207 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.778139 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 01:40:29.778207 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.778161 2553 factory.go:103] Registering Raw factory Jun 03 01:40:29.778207 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.778178 2553 manager.go:1196] Started watching for new ooms in manager Jun 03 01:40:29.778713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.778690 2553 manager.go:319] Starting recovery of all containers Jun 03 01:40:29.779341 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.779320 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-201.ec2.internal" not found Jun 03 01:40:29.781781 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.781760 2553 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:29.782502 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.782481 2553 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-201.ec2.internal\" not found" node="ip-10-0-136-201.ec2.internal" Jun 03 01:40:29.790466 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.790445 2553 manager.go:324] Recovery completed Jun 03 01:40:29.794571 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.794521 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-201.ec2.internal" not found Jun 03 01:40:29.794646 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.794596 2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 01:40:29.796722 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.796704 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientMemory" Jun 03 01:40:29.796822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.796740 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 01:40:29.796822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.796754 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientPID" Jun 03 01:40:29.797329 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.797317 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 01:40:29.797329 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.797329 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 01:40:29.797408 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.797348 2553 state_mem.go:36] "Initialized new in-memory state store" Jun 03 01:40:29.800417 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.800397 2553 policy_none.go:49] "None policy: Start" Jun 03 01:40:29.800417 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.800417 2553 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 01:40:29.800522 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.800427 2553 state_mem.go:35] "Initializing new in-memory state store" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839308 2553 manager.go:341] "Starting Device Plugin manager" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.839349 2553 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839363 2553 server.go:85] "Starting device plugin registration server" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839688 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839709 2553 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839800 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839893 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.839902 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.840368 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.840407 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:29.852625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.850559 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-201.ec2.internal" not found Jun 03 01:40:29.932902 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.932810 2553 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 01:40:29.934246 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.934224 2553 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 01:40:29.934376 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.934255 2553 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 01:40:29.934376 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.934276 2553 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 01:40:29.934376 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.934282 2553 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 01:40:29.934376 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.934360 2553 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 01:40:29.938697 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.938673 2553 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:29.940399 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.940384 2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 01:40:29.941255 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.941241 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientMemory" Jun 03 01:40:29.941309 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.941273 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 01:40:29.941309 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.941286 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientPID" Jun 03 01:40:29.941309 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.941310 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-201.ec2.internal" Jun 03 01:40:29.949571 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:29.949553 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-201.ec2.internal" Jun 03 01:40:29.949683 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.949580 2553 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-201.ec2.internal\": node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:29.966823 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:29.966794 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.035366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.035290 2553 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal"] Jun 03 01:40:30.035502 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.035422 2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 01:40:30.037256 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.037235 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientMemory" Jun 03 01:40:30.037335 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.037272 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 01:40:30.037335 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.037282 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientPID" Jun 03 01:40:30.038602 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.038590 2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 01:40:30.038791 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.038776 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.038831 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.038814 2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 01:40:30.039377 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.039362 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientMemory" Jun 03 01:40:30.039468 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.039389 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientMemory" Jun 03 01:40:30.039468 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.039415 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 01:40:30.039468 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.039425 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientPID" Jun 03 01:40:30.039468 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.039393 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 01:40:30.039621 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.039474 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientPID" Jun 03 01:40:30.040631 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.040617 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.040667 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.040642 2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 01:40:30.041414 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.041398 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientMemory" Jun 03 01:40:30.041495 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.041429 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 01:40:30.041495 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.041445 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeHasSufficientPID" Jun 03 01:40:30.066920 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.066888 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.068317 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.068303 2553 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-201.ec2.internal\" not found" node="ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.072731 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.072713 2553 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-201.ec2.internal\" not found" node="ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.077896 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.077872 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5760d5d3b69b8c00fa721bd2dc13a279-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal\" (UID: \"5760d5d3b69b8c00fa721bd2dc13a279\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.077989 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.077903 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d060eb54ad6b2dfc1cc515cf81694a9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-201.ec2.internal\" (UID: \"8d060eb54ad6b2dfc1cc515cf81694a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.077989 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.077922 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5760d5d3b69b8c00fa721bd2dc13a279-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal\" (UID: \"5760d5d3b69b8c00fa721bd2dc13a279\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.167168 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.167126 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.178618 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.178590 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5760d5d3b69b8c00fa721bd2dc13a279-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal\" (UID: \"5760d5d3b69b8c00fa721bd2dc13a279\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.178709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.178626 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5760d5d3b69b8c00fa721bd2dc13a279-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal\" (UID: \"5760d5d3b69b8c00fa721bd2dc13a279\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.178709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.178666 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d060eb54ad6b2dfc1cc515cf81694a9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-201.ec2.internal\" (UID: \"8d060eb54ad6b2dfc1cc515cf81694a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.178709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.178686 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5760d5d3b69b8c00fa721bd2dc13a279-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal\" (UID: \"5760d5d3b69b8c00fa721bd2dc13a279\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.178809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.178730 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5760d5d3b69b8c00fa721bd2dc13a279-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal\" (UID: \"5760d5d3b69b8c00fa721bd2dc13a279\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.178809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.178764 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d060eb54ad6b2dfc1cc515cf81694a9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-201.ec2.internal\" (UID: \"8d060eb54ad6b2dfc1cc515cf81694a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.268042 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.267962 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.368612 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.368570 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.372737 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.372720 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.378754 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.378736 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.468928 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.468885 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.569510 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.569427 2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-201.ec2.internal\" not found" Jun 03 01:40:30.654343 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.654311 2553 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:30.670147 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.670116 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 01:40:30.670271 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.670243 2553 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 01:40:30.670271 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.670258 2553 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 01:40:30.670271 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.670264 2553 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 01:40:30.677117 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.677099 2553 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.696663 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.696627 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 01:40:30.699111 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.699092 2553 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" Jun 03 01:40:30.707308 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.707290 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 01:40:30.741490 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.741464 2553 apiserver.go:52] "Watching apiserver" Jun 03 01:40:30.751295 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.751139 2553 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 01:40:30.752467 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.752443 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal","openshift-cluster-node-tuning-operator/tuned-4jnwc","openshift-dns/node-resolver-gv8d4","openshift-multus/multus-additional-cni-plugins-rd2nk","openshift-multus/network-metrics-daemon-66gkf","openshift-network-operator/iptables-alerter-qwtj4","openshift-ovn-kubernetes/ovnkube-node-475tz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw","openshift-image-registry/node-ca-t55ss","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal","openshift-multus/multus-r29r8","openshift-network-diagnostics/network-check-target-mv2xg","kube-system/konnectivity-agent-pjvzl"] Jun 03 01:40:30.755859 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.755834 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.757231 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.757214 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:30.757323 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.757285 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:30.757323 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.757307 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.757639 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.757613 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 01:35:29 +0000 UTC" deadline="2027-10-27 23:42:44.468144104 +0000 UTC" Jun 03 01:40:30.757639 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.757638 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12286h2m13.710508447s" Jun 03 01:40:30.758629 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.758616 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 01:40:30.758939 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.758928 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.759166 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.759137 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.759498 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.759483 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.760100 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.760080 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:30.760170 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.760130 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:30.761147 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761127 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vpv4k\"" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761204 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761218 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761335 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hv2mw\"" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761453 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761527 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761553 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 01:40:30.761884 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761627 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.762277 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761922 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 01:40:30.762277 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.761983 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.762277 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.762035 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4b4gr\"" Jun 03 01:40:30.762277 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.762120 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 01:40:30.762938 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.762922 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.763937 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.763919 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.764194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.764178 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.764503 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.764340 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 01:40:30.764991 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.764971 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tqcnl\"" Jun 03 01:40:30.764991 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.764987 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.765311 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.765289 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6nbmn\"" Jun 03 01:40:30.765375 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.765360 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.765430 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.765410 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 01:40:30.765666 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.765643 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.766689 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.766607 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.767232 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.766941 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8dvff\"" Jun 03 01:40:30.767600 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.767584 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 01:40:30.767684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.767636 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.767872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.767851 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 01:40:30.768721 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.768704 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.768889 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.768875 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 01:40:30.770684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.770667 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.770929 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.770913 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 01:40:30.771096 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.771082 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qjpvl\"" Jun 03 01:40:30.772009 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.771994 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.772151 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.772137 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 01:40:30.773760 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.773743 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 01:40:30.773837 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.773821 2553 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 01:40:30.774043 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.774027 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 01:40:30.774144 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.774116 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4nwjk\"" Jun 03 01:40:30.774400 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.774386 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rq42d\"" Jun 03 01:40:30.774592 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.774520 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 01:40:30.774592 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.774567 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 01:40:30.781758 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781733 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-conf-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.781846 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781760 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cnibin\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.781846 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781786 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.781846 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781822 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-kubelet\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.781846 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781816 2553 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 01:40:30.781846 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781862 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-ovn\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781886 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781920 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-socket-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781964 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-device-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.781996 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-lib-modules\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782021 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782036 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-cni-bin\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782050 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-cni-netd\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782070 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782068 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cni-binary-copy\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782113 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-k8s-cni-cncf-io\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782143 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-hostroot\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782182 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-host\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782206 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3da5575-9d26-4064-85da-7c11be51c8d4-serviceca\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782239 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782265 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-env-overrides\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782308 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-os-release\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782332 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba75e2df-06f0-4525-a113-4cf6d325fd05-tmp\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782355 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-system-cni-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782383 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysctl-conf\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782404 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-systemd\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gkxh\" (UniqueName: \"kubernetes.io/projected/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-kube-api-access-8gkxh\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782443 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-etc-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782477 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-sys-fs\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782551 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-cni-bin\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782590 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-tuned\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782617 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-systemd\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782631 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovnkube-script-lib\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782649 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rkzj\" (UniqueName: \"kubernetes.io/projected/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-kube-api-access-9rkzj\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782672 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-cnibin\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1e82b37-06ce-4455-924f-8c132622eae1-konnectivity-ca\") pod \"konnectivity-agent-pjvzl\" (UID: \"b1e82b37-06ce-4455-924f-8c132622eae1\") " pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782707 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-kubelet\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782730 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782751 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-cni-binary-copy\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782772 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-run\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782800 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-log-socket\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782818 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z44pt\" (UniqueName: \"kubernetes.io/projected/07783117-ea29-4d81-92c3-2c4426c42a0b-kube-api-access-z44pt\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfn2l\" (UniqueName: \"kubernetes.io/projected/1b755398-cf78-4497-82eb-76883882bd5d-kube-api-access-jfn2l\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782855 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x4zf\" (UniqueName: \"kubernetes.io/projected/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-kube-api-access-4x4zf\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.782997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782871 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k5wc\" (UniqueName: \"kubernetes.io/projected/ba75e2df-06f0-4525-a113-4cf6d325fd05-kube-api-access-5k5wc\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddmlj\" (UniqueName: \"kubernetes.io/projected/b11762dd-14ed-48d3-a346-a3997ab3567b-kube-api-access-ddmlj\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782915 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-systemd-units\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782941 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-var-lib-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782956 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07783117-ea29-4d81-92c3-2c4426c42a0b-iptables-alerter-script\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782983 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.782999 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-sys\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783014 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-tmp-dir\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783028 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-slash\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783049 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783091 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-daemon-config\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783124 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-multus-certs\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783152 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysctl-d\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783176 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-hosts-file\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783199 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-netns\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783224 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-cni-multus\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.783658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783247 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysconfig\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-run-ovn-kubernetes\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783300 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07783117-ea29-4d81-92c3-2c4426c42a0b-host-slash\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783342 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drwf2\" (UniqueName: \"kubernetes.io/projected/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-kube-api-access-drwf2\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783371 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-cni-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783387 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-modprobe-d\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783401 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-node-log\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783417 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovn-node-metrics-cert\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783432 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-registration-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783458 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-system-cni-dir\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783473 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-etc-kubernetes\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783486 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-kubernetes\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783499 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-var-lib-kubelet\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783518 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1e82b37-06ce-4455-924f-8c132622eae1-agent-certs\") pod \"konnectivity-agent-pjvzl\" (UID: \"b1e82b37-06ce-4455-924f-8c132622eae1\") " pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783562 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85448\" (UniqueName: \"kubernetes.io/projected/c3da5575-9d26-4064-85da-7c11be51c8d4-kube-api-access-85448\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783602 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-run-netns\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.784122 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783631 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-os-release\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.784595 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783648 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.784595 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3da5575-9d26-4064-85da-7c11be51c8d4-host\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.784595 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783717 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovnkube-config\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.784595 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.783731 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-socket-dir-parent\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.801502 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.801479 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zrtsg" Jun 03 01:40:30.808964 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.808927 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zrtsg" Jun 03 01:40:30.884161 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884135 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cnibin\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.884280 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884174 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.884280 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884200 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-kubelet\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.884280 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884233 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cnibin\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.884280 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884250 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-kubelet\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884295 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884324 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-ovn\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884341 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884355 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-socket-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884366 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-ovn\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884369 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-device-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884410 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-lib-modules\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884425 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:30.884480 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884427 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.884508 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884550 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-cni-bin\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884556 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-lib-modules\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884563 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-device-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884597 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-cni-netd\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884609 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-cni-bin\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.884626 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:31.384595836 +0000 UTC m=+2.039239122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884636 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-cni-netd\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884657 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cni-binary-copy\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884677 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-k8s-cni-cncf-io\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884693 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-hostroot\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884716 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-host\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884745 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-k8s-cni-cncf-io\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884751 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-hostroot\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884761 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-host\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884768 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.884864 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884776 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3da5575-9d26-4064-85da-7c11be51c8d4-serviceca\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884518 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-socket-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884854 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884891 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-env-overrides\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884896 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884919 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-os-release\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884946 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba75e2df-06f0-4525-a113-4cf6d325fd05-tmp\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884970 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-system-cni-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.884995 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysctl-conf\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885017 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-systemd\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885041 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8gkxh\" (UniqueName: \"kubernetes.io/projected/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-kube-api-access-8gkxh\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885045 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-os-release\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885068 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-etc-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885093 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-sys-fs\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885117 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-cni-bin\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885141 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-tuned\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885165 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-systemd\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.885485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885174 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysctl-conf\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885189 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovnkube-script-lib\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885199 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3da5575-9d26-4064-85da-7c11be51c8d4-serviceca\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885212 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cni-binary-copy\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885213 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9rkzj\" (UniqueName: \"kubernetes.io/projected/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-kube-api-access-9rkzj\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885226 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-system-cni-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885252 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-cnibin\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885256 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-cni-bin\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885275 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-systemd\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885277 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-sys-fs\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885291 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1e82b37-06ce-4455-924f-8c132622eae1-konnectivity-ca\") pod \"konnectivity-agent-pjvzl\" (UID: \"b1e82b37-06ce-4455-924f-8c132622eae1\") " pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885313 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-systemd\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885318 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-kubelet\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885365 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885389 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-cni-binary-copy\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885396 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-etc-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885412 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-run\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885444 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-log-socket\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.886250 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885450 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-cnibin\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885454 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885465 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-kubelet\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885469 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z44pt\" (UniqueName: \"kubernetes.io/projected/07783117-ea29-4d81-92c3-2c4426c42a0b-kube-api-access-z44pt\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885506 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jfn2l\" (UniqueName: \"kubernetes.io/projected/1b755398-cf78-4497-82eb-76883882bd5d-kube-api-access-jfn2l\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885510 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-run\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885513 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885585 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4x4zf\" (UniqueName: \"kubernetes.io/projected/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-kube-api-access-4x4zf\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885611 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5k5wc\" (UniqueName: \"kubernetes.io/projected/ba75e2df-06f0-4525-a113-4cf6d325fd05-kube-api-access-5k5wc\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885632 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmlj\" (UniqueName: \"kubernetes.io/projected/b11762dd-14ed-48d3-a346-a3997ab3567b-kube-api-access-ddmlj\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885654 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-systemd-units\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885677 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-var-lib-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885706 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07783117-ea29-4d81-92c3-2c4426c42a0b-iptables-alerter-script\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885725 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-log-socket\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885730 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885761 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-sys\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885791 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-tmp-dir\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.887293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885814 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-slash\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885831 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.885845 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-daemon-config\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886040 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1e82b37-06ce-4455-924f-8c132622eae1-konnectivity-ca\") pod \"konnectivity-agent-pjvzl\" (UID: \"b1e82b37-06ce-4455-924f-8c132622eae1\") " pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886145 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-multus-certs\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886176 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysctl-d\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886192 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-multus-certs\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886198 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-hosts-file\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886229 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-netns\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886250 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-hosts-file\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886257 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-cni-multus\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886259 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-daemon-config\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886149 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-cni-binary-copy\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886283 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysconfig\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886307 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-run-netns\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886314 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-run-ovn-kubernetes\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886328 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysconfig\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886342 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07783117-ea29-4d81-92c3-2c4426c42a0b-host-slash\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.888097 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886352 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-sysctl-d\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886360 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-drwf2\" (UniqueName: \"kubernetes.io/projected/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-kube-api-access-drwf2\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886378 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-cni-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886383 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-sys\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886396 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-modprobe-d\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886422 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-slash\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886423 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07783117-ea29-4d81-92c3-2c4426c42a0b-host-slash\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886440 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-node-log\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886469 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-modprobe-d\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886471 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovn-node-metrics-cert\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886478 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-systemd-units\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886493 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-registration-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886510 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-node-log\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886520 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-var-lib-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886523 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1b755398-cf78-4497-82eb-76883882bd5d-registration-dir\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886568 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-system-cni-dir\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886610 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-cni-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.888921 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886617 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-etc-kubernetes\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886645 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-kubernetes\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886647 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-run-openvswitch\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886376 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-host-var-lib-cni-multus\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886715 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886729 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-tmp-dir\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886788 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-var-lib-kubelet\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886833 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-var-lib-kubelet\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886843 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1e82b37-06ce-4455-924f-8c132622eae1-agent-certs\") pod \"konnectivity-agent-pjvzl\" (UID: \"b1e82b37-06ce-4455-924f-8c132622eae1\") " pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886875 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-85448\" (UniqueName: \"kubernetes.io/projected/c3da5575-9d26-4064-85da-7c11be51c8d4-kube-api-access-85448\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886900 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-run-netns\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886926 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-os-release\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886983 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887058 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3da5575-9d26-4064-85da-7c11be51c8d4-host\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887086 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovnkube-config\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887115 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-socket-dir-parent\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887140 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-conf-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.889615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887182 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovnkube-script-lib\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887218 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-env-overrides\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887275 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-conf-dir\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887315 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-os-release\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887362 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-system-cni-dir\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887402 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-etc-kubernetes\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887445 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-kubernetes\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.886877 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-run-ovn-kubernetes\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887486 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-host-run-netns\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887557 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-multus-socket-dir-parent\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887600 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3da5575-9d26-4064-85da-7c11be51c8d4-host\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.887991 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovnkube-config\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.888126 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.888370 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07783117-ea29-4d81-92c3-2c4426c42a0b-iptables-alerter-script\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.890179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.889526 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba75e2df-06f0-4525-a113-4cf6d325fd05-tmp\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.890709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.890227 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1e82b37-06ce-4455-924f-8c132622eae1-agent-certs\") pod \"konnectivity-agent-pjvzl\" (UID: \"b1e82b37-06ce-4455-924f-8c132622eae1\") " pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.890709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.890306 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba75e2df-06f0-4525-a113-4cf6d325fd05-etc-tuned\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.890709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.890404 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-ovn-node-metrics-cert\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.891656 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.891625 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:40:30.891748 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.891673 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:40:30.891748 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.891686 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:30.891821 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:30.891756 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:40:31.39173977 +0000 UTC m=+2.046383040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:30.895546 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.895488 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z44pt\" (UniqueName: \"kubernetes.io/projected/07783117-ea29-4d81-92c3-2c4426c42a0b-kube-api-access-z44pt\") pod \"iptables-alerter-qwtj4\" (UID: \"07783117-ea29-4d81-92c3-2c4426c42a0b\") " pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:30.896520 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.896473 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddmlj\" (UniqueName: \"kubernetes.io/projected/b11762dd-14ed-48d3-a346-a3997ab3567b-kube-api-access-ddmlj\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:30.897171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.897097 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k5wc\" (UniqueName: \"kubernetes.io/projected/ba75e2df-06f0-4525-a113-4cf6d325fd05-kube-api-access-5k5wc\") pod \"tuned-4jnwc\" (UID: \"ba75e2df-06f0-4525-a113-4cf6d325fd05\") " pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.897171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.897107 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rkzj\" (UniqueName: \"kubernetes.io/projected/b8ae6056-f1d4-4ec0-8d77-3981347bcbd5-kube-api-access-9rkzj\") pod \"ovnkube-node-475tz\" (UID: \"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:30.897360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.897241 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfn2l\" (UniqueName: \"kubernetes.io/projected/1b755398-cf78-4497-82eb-76883882bd5d-kube-api-access-jfn2l\") pod \"aws-ebs-csi-driver-node-kwpfw\" (UID: \"1b755398-cf78-4497-82eb-76883882bd5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:30.897360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.897260 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x4zf\" (UniqueName: \"kubernetes.io/projected/0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a-kube-api-access-4x4zf\") pod \"multus-r29r8\" (UID: \"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a\") " pod="openshift-multus/multus-r29r8" Jun 03 01:40:30.897360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.897265 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85448\" (UniqueName: \"kubernetes.io/projected/c3da5575-9d26-4064-85da-7c11be51c8d4-kube-api-access-85448\") pod \"node-ca-t55ss\" (UID: \"c3da5575-9d26-4064-85da-7c11be51c8d4\") " pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:30.898402 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.898373 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gkxh\" (UniqueName: \"kubernetes.io/projected/ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4-kube-api-access-8gkxh\") pod \"node-resolver-gv8d4\" (UID: \"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4\") " pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:30.898628 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.898611 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drwf2\" (UniqueName: \"kubernetes.io/projected/e1e6ff2f-3769-40c8-b42c-ba311fd1deca-kube-api-access-drwf2\") pod \"multus-additional-cni-plugins-rd2nk\" (UID: \"e1e6ff2f-3769-40c8-b42c-ba311fd1deca\") " pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:30.920504 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.920481 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" Jun 03 01:40:30.925444 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:30.925387 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d060eb54ad6b2dfc1cc515cf81694a9.slice/crio-0dd074606825588d846186dc514c7f62de6422658c62073b5a3a607dc3184536 WatchSource:0}: Error finding container 0dd074606825588d846186dc514c7f62de6422658c62073b5a3a607dc3184536: Status 404 returned error can't find the container with id 0dd074606825588d846186dc514c7f62de6422658c62073b5a3a607dc3184536 Jun 03 01:40:30.925790 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:30.925764 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5760d5d3b69b8c00fa721bd2dc13a279.slice/crio-6240c475f714745b94d48ef3d8b64893c6e6e73c9194db02d7ca9174d80e0de6 WatchSource:0}: Error finding container 6240c475f714745b94d48ef3d8b64893c6e6e73c9194db02d7ca9174d80e0de6: Status 404 returned error can't find the container with id 6240c475f714745b94d48ef3d8b64893c6e6e73c9194db02d7ca9174d80e0de6 Jun 03 01:40:30.929950 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.929929 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:30.930097 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:30.930075 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba75e2df_06f0_4525_a113_4cf6d325fd05.slice/crio-02cabb124984856b755464f1166c715afa128d5c21bc997e2bc1375bfc26f646 WatchSource:0}: Error finding container 02cabb124984856b755464f1166c715afa128d5c21bc997e2bc1375bfc26f646: Status 404 returned error can't find the container with id 02cabb124984856b755464f1166c715afa128d5c21bc997e2bc1375bfc26f646 Jun 03 01:40:30.931025 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.930908 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 01:40:30.937394 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:30.937362 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e82b37_06ce_4455_924f_8c132622eae1.slice/crio-04e820b8e5a23f9da19999d3813cb3e7d8689217b08c2c55b152f9441f5b41f6 WatchSource:0}: Error finding container 04e820b8e5a23f9da19999d3813cb3e7d8689217b08c2c55b152f9441f5b41f6: Status 404 returned error can't find the container with id 04e820b8e5a23f9da19999d3813cb3e7d8689217b08c2c55b152f9441f5b41f6 Jun 03 01:40:30.938491 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.938443 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" event={"ID":"ba75e2df-06f0-4525-a113-4cf6d325fd05","Type":"ContainerStarted","Data":"02cabb124984856b755464f1166c715afa128d5c21bc997e2bc1375bfc26f646"} Jun 03 01:40:30.939626 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.939605 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" event={"ID":"5760d5d3b69b8c00fa721bd2dc13a279","Type":"ContainerStarted","Data":"6240c475f714745b94d48ef3d8b64893c6e6e73c9194db02d7ca9174d80e0de6"} Jun 03 01:40:30.940656 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:30.940638 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" event={"ID":"8d060eb54ad6b2dfc1cc515cf81694a9","Type":"ContainerStarted","Data":"0dd074606825588d846186dc514c7f62de6422658c62073b5a3a607dc3184536"} Jun 03 01:40:31.086975 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.086942 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t55ss" Jun 03 01:40:31.094402 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.094373 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3da5575_9d26_4064_85da_7c11be51c8d4.slice/crio-8574c61b5aab51301fe929b5e66103e230223e4b78998012272946166b332829 WatchSource:0}: Error finding container 8574c61b5aab51301fe929b5e66103e230223e4b78998012272946166b332829: Status 404 returned error can't find the container with id 8574c61b5aab51301fe929b5e66103e230223e4b78998012272946166b332829 Jun 03 01:40:31.107269 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.107243 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gv8d4" Jun 03 01:40:31.113315 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.113289 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1bd7c9_7d88_4354_9d35_1fbfa1531cf4.slice/crio-2b88ba51fabcd62622a719f6e1f82ea85c19a3346f045fd6ba7192340455a4c5 WatchSource:0}: Error finding container 2b88ba51fabcd62622a719f6e1f82ea85c19a3346f045fd6ba7192340455a4c5: Status 404 returned error can't find the container with id 2b88ba51fabcd62622a719f6e1f82ea85c19a3346f045fd6ba7192340455a4c5 Jun 03 01:40:31.120230 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.120207 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:31.126610 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.126577 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ae6056_f1d4_4ec0_8d77_3981347bcbd5.slice/crio-7d9dd20fcbbfd1be9819176fb97ad09f2a5cdbce6cf4687ccafa1593ec85ba46 WatchSource:0}: Error finding container 7d9dd20fcbbfd1be9819176fb97ad09f2a5cdbce6cf4687ccafa1593ec85ba46: Status 404 returned error can't find the container with id 7d9dd20fcbbfd1be9819176fb97ad09f2a5cdbce6cf4687ccafa1593ec85ba46 Jun 03 01:40:31.134683 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.134665 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qwtj4" Jun 03 01:40:31.142109 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.142056 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07783117_ea29_4d81_92c3_2c4426c42a0b.slice/crio-dbdcf659f7635588789f7eac35c2dcf6a37eb052e7e67d86ed1b60f82af66b44 WatchSource:0}: Error finding container dbdcf659f7635588789f7eac35c2dcf6a37eb052e7e67d86ed1b60f82af66b44: Status 404 returned error can't find the container with id dbdcf659f7635588789f7eac35c2dcf6a37eb052e7e67d86ed1b60f82af66b44 Jun 03 01:40:31.156002 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.155977 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" Jun 03 01:40:31.162258 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.162237 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b755398_cf78_4497_82eb_76883882bd5d.slice/crio-3ee6007d344ea236a6c68954cd7be49ec2c18aff9703d8a9f49bf2787a318796 WatchSource:0}: Error finding container 3ee6007d344ea236a6c68954cd7be49ec2c18aff9703d8a9f49bf2787a318796: Status 404 returned error can't find the container with id 3ee6007d344ea236a6c68954cd7be49ec2c18aff9703d8a9f49bf2787a318796 Jun 03 01:40:31.174046 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.174022 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" Jun 03 01:40:31.180445 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.180421 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e6ff2f_3769_40c8_b42c_ba311fd1deca.slice/crio-0745375e73ddf8b209d2a6ab0ee9ebe7d526bbc6e32dd9b65d3c6f159a6f261f WatchSource:0}: Error finding container 0745375e73ddf8b209d2a6ab0ee9ebe7d526bbc6e32dd9b65d3c6f159a6f261f: Status 404 returned error can't find the container with id 0745375e73ddf8b209d2a6ab0ee9ebe7d526bbc6e32dd9b65d3c6f159a6f261f Jun 03 01:40:31.188980 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.188956 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r29r8" Jun 03 01:40:31.196219 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:40:31.196184 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea6fc9a_0e75_4de7_9c7b_9b76ca3c226a.slice/crio-1381ed94a2acea1bfe72c9e5b47821750085061276080042717a092d330eb205 WatchSource:0}: Error finding container 1381ed94a2acea1bfe72c9e5b47821750085061276080042717a092d330eb205: Status 404 returned error can't find the container with id 1381ed94a2acea1bfe72c9e5b47821750085061276080042717a092d330eb205 Jun 03 01:40:31.249638 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.249606 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7v2pd"] Jun 03 01:40:31.249942 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.249927 2553 predicate.go:212] "Predicate failed on Pod" pod="kube-system/global-pull-secret-syncer-7v2pd" err="Predicate NodeAffinity failed: node(s) didn't match Pod's node affinity/selector" Jun 03 01:40:31.249978 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.249943 2553 kubelet.go:2420] "Pod admission denied" podUID="31af7a02-9c55-4765-95d6-8ab1ed598d7b" pod="kube-system/global-pull-secret-syncer-7v2pd" reason="NodeAffinity" message="Predicate NodeAffinity failed: node(s) didn't match Pod's node affinity/selector" Jun 03 01:40:31.269257 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.269210 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7v2pd" podStartSLOduration=0.26919672 podStartE2EDuration="269.19672ms" podCreationTimestamp="2026-06-03 01:40:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:40:31.268994801 +0000 UTC m=+1.923638081" watchObservedRunningTime="2026-06-03 01:40:31.26919672 +0000 UTC m=+1.923840000" Jun 03 01:40:31.281651 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.281628 2553 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:31.289811 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.289775 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.289954 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.289906 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-dbus\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.290016 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.289955 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-kubelet-config\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.391396 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.391309 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.391396 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.391379 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:31.391664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.391422 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-dbus\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.391664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.391464 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-kubelet-config\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.391664 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.391490 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:31.391664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.391574 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-dbus\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.391664 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.391589 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret podName:31af7a02-9c55-4765-95d6-8ab1ed598d7b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:31.891562572 +0000 UTC m=+2.546205845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret") pod "global-pull-secret-syncer-7v2pd" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:31.391664 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.391638 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:31.391989 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.391707 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:32.391680786 +0000 UTC m=+3.046324051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:31.391989 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.391639 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-kubelet-config\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.492450 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.492414 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:31.492650 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.492598 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:40:31.492650 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.492618 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:40:31.492650 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.492631 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:31.492814 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.492684 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:40:32.492666487 +0000 UTC m=+3.147309751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:31.751952 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.751868 2553 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:31.810023 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.809988 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 01:35:30 +0000 UTC" deadline="2028-03-18 22:59:56.127936102 +0000 UTC" Jun 03 01:40:31.810023 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.810018 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15717h19m24.317920482s" Jun 03 01:40:31.879293 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.879263 2553 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:31.896814 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.896778 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:31.897006 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.896989 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:31.897080 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:31.897060 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret podName:31af7a02-9c55-4765-95d6-8ab1ed598d7b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:32.897040802 +0000 UTC m=+3.551684082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret") pod "global-pull-secret-syncer-7v2pd" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:31.971164 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.971078 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gv8d4" event={"ID":"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4","Type":"ContainerStarted","Data":"2b88ba51fabcd62622a719f6e1f82ea85c19a3346f045fd6ba7192340455a4c5"} Jun 03 01:40:31.987840 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.987778 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t55ss" event={"ID":"c3da5575-9d26-4064-85da-7c11be51c8d4","Type":"ContainerStarted","Data":"8574c61b5aab51301fe929b5e66103e230223e4b78998012272946166b332829"} Jun 03 01:40:31.993413 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.993347 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjvzl" event={"ID":"b1e82b37-06ce-4455-924f-8c132622eae1","Type":"ContainerStarted","Data":"04e820b8e5a23f9da19999d3813cb3e7d8689217b08c2c55b152f9441f5b41f6"} Jun 03 01:40:31.997924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.997658 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-kubelet-config\") pod \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " Jun 03 01:40:31.997924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.997713 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-dbus\") pod \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " Jun 03 01:40:31.997924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.997771 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-kubelet-config" (OuterVolumeSpecName: "kubelet-config") pod "31af7a02-9c55-4765-95d6-8ab1ed598d7b" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b"). InnerVolumeSpecName "kubelet-config". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Jun 03 01:40:31.997924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.997817 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-dbus" (OuterVolumeSpecName: "dbus") pod "31af7a02-9c55-4765-95d6-8ab1ed598d7b" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b"). InnerVolumeSpecName "dbus". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Jun 03 01:40:31.997924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.997884 2553 reconciler_common.go:299] "Volume detached for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-dbus\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:40:31.997924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:31.997901 2553 reconciler_common.go:299] "Volume detached for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31af7a02-9c55-4765-95d6-8ab1ed598d7b-kubelet-config\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:40:32.001658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.001571 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r29r8" event={"ID":"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a","Type":"ContainerStarted","Data":"1381ed94a2acea1bfe72c9e5b47821750085061276080042717a092d330eb205"} Jun 03 01:40:32.012489 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.012385 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" event={"ID":"1b755398-cf78-4497-82eb-76883882bd5d","Type":"ContainerStarted","Data":"3ee6007d344ea236a6c68954cd7be49ec2c18aff9703d8a9f49bf2787a318796"} Jun 03 01:40:32.025226 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.025187 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerStarted","Data":"0745375e73ddf8b209d2a6ab0ee9ebe7d526bbc6e32dd9b65d3c6f159a6f261f"} Jun 03 01:40:32.044913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.044877 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qwtj4" event={"ID":"07783117-ea29-4d81-92c3-2c4426c42a0b","Type":"ContainerStarted","Data":"dbdcf659f7635588789f7eac35c2dcf6a37eb052e7e67d86ed1b60f82af66b44"} Jun 03 01:40:32.067481 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.067442 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"7d9dd20fcbbfd1be9819176fb97ad09f2a5cdbce6cf4687ccafa1593ec85ba46"} Jun 03 01:40:32.400378 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.400254 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:32.400601 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.400418 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:32.400601 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.400482 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:34.400462119 +0000 UTC m=+5.055105391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:32.501463 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.501423 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:32.501642 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.501629 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:40:32.501695 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.501655 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:40:32.501695 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.501669 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:32.501759 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.501732 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:40:34.501708662 +0000 UTC m=+5.156351925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:32.810877 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.810757 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 01:35:30 +0000 UTC" deadline="2028-02-05 07:09:14.399536735 +0000 UTC" Jun 03 01:40:32.810877 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.810795 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14693h28m41.588744634s" Jun 03 01:40:32.902224 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.902183 2553 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 01:40:32.905440 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.905411 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:32.905630 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.905613 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:32.905700 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.905684 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret podName:31af7a02-9c55-4765-95d6-8ab1ed598d7b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:34.905665564 +0000 UTC m=+5.560308836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret") pod "global-pull-secret-syncer-7v2pd" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:32.935294 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.935260 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:32.935464 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:32.935261 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:32.935464 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.935415 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:32.935611 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:32.935467 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:34.418651 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:34.418616 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:34.419174 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.418785 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:34.419174 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.418852 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:38.418834142 +0000 UTC m=+9.073477399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:34.519587 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:34.519546 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:34.519784 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.519769 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:40:34.519847 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.519789 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:40:34.519847 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.519803 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:34.519947 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.519864 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:40:38.519844044 +0000 UTC m=+9.174487325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:34.923416 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:34.922757 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:34.923416 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.922977 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:34.923416 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.923046 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret podName:31af7a02-9c55-4765-95d6-8ab1ed598d7b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:38.923027254 +0000 UTC m=+9.577670529 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret") pod "global-pull-secret-syncer-7v2pd" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:34.934779 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:34.934741 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:34.934951 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.934893 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:34.935324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:34.935302 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:34.935427 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:34.935398 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:36.935844 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:36.935143 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:36.935844 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:36.935289 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:36.935844 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:36.935705 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:36.935844 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:36.935794 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:38.455033 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:38.454989 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:38.455506 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.455158 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:38.455506 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.455223 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:46.455204263 +0000 UTC m=+17.109847521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:38.555794 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:38.555761 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:38.555997 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.555974 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:40:38.555997 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.555998 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:40:38.556143 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.556011 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:38.556143 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.556074 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:40:46.55605348 +0000 UTC m=+17.210696757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:38.935813 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:38.935087 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:38.935813 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.935231 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:38.935813 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:38.935632 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:38.935813 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.935714 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:38.960892 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:38.960287 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") pod \"global-pull-secret-syncer-7v2pd\" (UID: \"31af7a02-9c55-4765-95d6-8ab1ed598d7b\") " pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:38.960892 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.960477 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:38.960892 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:38.960546 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret podName:31af7a02-9c55-4765-95d6-8ab1ed598d7b nodeName:}" failed. No retries permitted until 2026-06-03 01:40:46.960517725 +0000 UTC m=+17.615160985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret") pod "global-pull-secret-syncer-7v2pd" (UID: "31af7a02-9c55-4765-95d6-8ab1ed598d7b") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:39.958949 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.958569 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kube-system/global-pull-secret-syncer-7v2pd"] Jun 03 01:40:39.958949 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.958668 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7v2pd" Jun 03 01:40:39.962129 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.962099 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kube-system/global-pull-secret-syncer-7v2pd"] Jun 03 01:40:39.963757 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.963721 2553 status_manager.go:895] "Failed to get status for pod" podUID="31af7a02-9c55-4765-95d6-8ab1ed598d7b" pod="kube-system/global-pull-secret-syncer-7v2pd" err="pods \"global-pull-secret-syncer-7v2pd\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kube-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:40:39.982253 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.982224 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-49j2x"] Jun 03 01:40:39.984189 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.984159 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:39.984316 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:39.984248 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:39.986742 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:39.986602 2553 status_manager.go:895] "Failed to get status for pod" podUID="31af7a02-9c55-4765-95d6-8ab1ed598d7b" pod="kube-system/global-pull-secret-syncer-7v2pd" err="pods \"global-pull-secret-syncer-7v2pd\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kube-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:40:40.071647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.071435 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b50f9317-b857-416c-b66a-1d7c38aa3c76-kubelet-config\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.071647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.071489 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b50f9317-b857-416c-b66a-1d7c38aa3c76-dbus\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.071647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.071545 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.071647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.071616 2553 reconciler_common.go:299] "Volume detached for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31af7a02-9c55-4765-95d6-8ab1ed598d7b-original-pull-secret\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:40:40.171938 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.171897 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b50f9317-b857-416c-b66a-1d7c38aa3c76-kubelet-config\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.172098 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.171950 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b50f9317-b857-416c-b66a-1d7c38aa3c76-dbus\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.172098 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.171987 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.172098 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.172035 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b50f9317-b857-416c-b66a-1d7c38aa3c76-kubelet-config\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.172238 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:40.172124 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:40.172238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.172126 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b50f9317-b857-416c-b66a-1d7c38aa3c76-dbus\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.172238 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:40.172173 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret podName:b50f9317-b857-416c-b66a-1d7c38aa3c76 nodeName:}" failed. No retries permitted until 2026-06-03 01:40:40.672159225 +0000 UTC m=+11.326802484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret") pod "global-pull-secret-syncer-49j2x" (UID: "b50f9317-b857-416c-b66a-1d7c38aa3c76") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:40.675869 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.675819 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:40.676031 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:40.675934 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:40.676031 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:40.675988 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret podName:b50f9317-b857-416c-b66a-1d7c38aa3c76 nodeName:}" failed. No retries permitted until 2026-06-03 01:40:41.675974934 +0000 UTC m=+12.330618192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret") pod "global-pull-secret-syncer-49j2x" (UID: "b50f9317-b857-416c-b66a-1d7c38aa3c76") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:40.935342 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.935255 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:40.935523 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:40.935255 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:40.935523 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:40.935374 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:40.935523 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:40.935486 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:41.683332 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:41.683292 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:41.683779 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:41.683442 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:41.683779 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:41.683517 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret podName:b50f9317-b857-416c-b66a-1d7c38aa3c76 nodeName:}" failed. No retries permitted until 2026-06-03 01:40:43.68349881 +0000 UTC m=+14.338142067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret") pod "global-pull-secret-syncer-49j2x" (UID: "b50f9317-b857-416c-b66a-1d7c38aa3c76") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:41.934720 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:41.934637 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:41.934862 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:41.934755 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:42.935195 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:42.935156 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:42.935696 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:42.935174 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:42.935696 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:42.935277 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:42.935696 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:42.935366 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:43.700139 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:43.700099 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:43.700326 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:43.700277 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:43.700391 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:43.700345 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret podName:b50f9317-b857-416c-b66a-1d7c38aa3c76 nodeName:}" failed. No retries permitted until 2026-06-03 01:40:47.70031913 +0000 UTC m=+18.354962406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret") pod "global-pull-secret-syncer-49j2x" (UID: "b50f9317-b857-416c-b66a-1d7c38aa3c76") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:43.935633 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:43.935098 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:43.935633 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:43.935235 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:44.934519 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:44.934480 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:44.934800 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:44.934480 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:44.934800 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:44.934625 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:44.934800 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:44.934664 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:45.934929 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:45.934895 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:45.935301 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:45.934996 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:46.519607 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:46.519566 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:46.519826 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.519723 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:46.519826 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.519803 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:41:02.519782411 +0000 UTC m=+33.174425669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:40:46.619991 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:46.619956 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:46.620186 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.620154 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:40:46.620186 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.620183 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:40:46.620279 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.620198 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:46.620279 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.620254 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:41:02.620238759 +0000 UTC m=+33.274882018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:40:46.935062 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:46.934973 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:46.935516 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.935110 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:46.935516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:46.934973 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:46.935649 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:46.935624 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:47.726239 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:47.726195 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:47.726432 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:47.726361 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:47.726483 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:47.726456 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret podName:b50f9317-b857-416c-b66a-1d7c38aa3c76 nodeName:}" failed. No retries permitted until 2026-06-03 01:40:55.726434581 +0000 UTC m=+26.381077862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret") pod "global-pull-secret-syncer-49j2x" (UID: "b50f9317-b857-416c-b66a-1d7c38aa3c76") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:47.935288 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:47.935251 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:47.935769 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:47.935372 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:48.935116 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:48.935081 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:48.935312 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:48.935129 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:48.935312 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:48.935193 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:48.935693 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:48.935355 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:49.935925 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:49.935889 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:49.936388 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:49.935983 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:50.934767 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:50.934522 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:50.934899 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:50.934871 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:50.935226 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:50.935207 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:50.935319 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:50.935300 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:51.113822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.113583 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" event={"ID":"8d060eb54ad6b2dfc1cc515cf81694a9","Type":"ContainerStarted","Data":"f2ba58980c43af232d395eacf03086c130c2178ea71203f46ba8784c21319c4f"} Jun 03 01:40:51.122244 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.122211 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r29r8" event={"ID":"0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a","Type":"ContainerStarted","Data":"9c117b88b4d1431feb4525580e46e00b8a3bc84c9ebd2ddf5e695a1e19bf5bb5"} Jun 03 01:40:51.125645 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.125595 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" event={"ID":"ba75e2df-06f0-4525-a113-4cf6d325fd05","Type":"ContainerStarted","Data":"0c646a6bc4389f32d04d144cc5ba608e01911eceda458be37e50b3eda4d4ace0"} Jun 03 01:40:51.127193 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.127170 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"7bbdab459a913b21219a8bd18d4571c8246f1f6ff407728fdafcd7a4f91508b7"} Jun 03 01:40:51.127284 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.127197 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"c6994a41ff099adcd7a5d760a603e59c266d1214ca3ac8c9bbe631cb290be223"} Jun 03 01:40:51.127284 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.127207 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"7815c3d54d583f04bf65681a4610ca86d11417b487f4d718035325ffc1e89013"} Jun 03 01:40:51.127881 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.127848 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-201.ec2.internal" podStartSLOduration=21.127836183 podStartE2EDuration="21.127836183s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:40:51.127387753 +0000 UTC m=+21.782031043" watchObservedRunningTime="2026-06-03 01:40:51.127836183 +0000 UTC m=+21.782479463" Jun 03 01:40:51.143102 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.143058 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-r29r8" podStartSLOduration=1.75880285 podStartE2EDuration="21.143046172s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.197574027 +0000 UTC m=+1.852217285" lastFinishedPulling="2026-06-03 01:40:50.581817349 +0000 UTC m=+21.236460607" observedRunningTime="2026-06-03 01:40:51.142577558 +0000 UTC m=+21.797220839" watchObservedRunningTime="2026-06-03 01:40:51.143046172 +0000 UTC m=+21.797689452" Jun 03 01:40:51.934819 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:51.934647 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:51.934995 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:51.934901 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:52.060829 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.060804 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 01:40:52.130600 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.130564 2553 generic.go:358] "Generic (PLEG): container finished" podID="5760d5d3b69b8c00fa721bd2dc13a279" containerID="d9fa87f15714c19abb3b432f248c978946c7972e951ceb0da0dbecafb36f149c" exitCode=0 Jun 03 01:40:52.131307 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.130635 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" event={"ID":"5760d5d3b69b8c00fa721bd2dc13a279","Type":"ContainerDied","Data":"d9fa87f15714c19abb3b432f248c978946c7972e951ceb0da0dbecafb36f149c"} Jun 03 01:40:52.132190 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.132163 2553 generic.go:358] "Generic (PLEG): container finished" podID="e1e6ff2f-3769-40c8-b42c-ba311fd1deca" containerID="8aff3b2f6431ef43436f22e49b5f9113c5e39f345616b92f036f036fa1a33132" exitCode=0 Jun 03 01:40:52.132308 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.132229 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerDied","Data":"8aff3b2f6431ef43436f22e49b5f9113c5e39f345616b92f036f036fa1a33132"} Jun 03 01:40:52.133588 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.133563 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qwtj4" event={"ID":"07783117-ea29-4d81-92c3-2c4426c42a0b","Type":"ContainerStarted","Data":"9784096f2de686b04324a39b6d53acc25fa4364fdf6423df1810f23e79966567"} Jun 03 01:40:52.136466 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.136441 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"a07548e0af6e6cd3c97675dba6bba62c5ee595af9d1b956d423da265c18d6c68"} Jun 03 01:40:52.136578 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.136495 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"d5de7beaef184df71e7780574723af7f089489b49e03f4e633cc11ce324c5d2f"} Jun 03 01:40:52.136578 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.136510 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"1d14b25bdf3b726f24126fd01e67a40703a79146ca75cf2dd3b491827dec7437"} Jun 03 01:40:52.137633 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.137608 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gv8d4" event={"ID":"ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4","Type":"ContainerStarted","Data":"200d91051d2a847d54b7346c40978855310cec121f4200017df6d159bda6e398"} Jun 03 01:40:52.138941 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.138926 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t55ss" event={"ID":"c3da5575-9d26-4064-85da-7c11be51c8d4","Type":"ContainerStarted","Data":"6ccc13528e6a585486c069c455769604d24f7a0646d7664c3d298463755712ed"} Jun 03 01:40:52.140184 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.140162 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjvzl" event={"ID":"b1e82b37-06ce-4455-924f-8c132622eae1","Type":"ContainerStarted","Data":"71b9c8a6328abb53a7d06fc1488e7d8ff4ac9b4eeda84e989e5611beff9e1316"} Jun 03 01:40:52.141623 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.141601 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" event={"ID":"1b755398-cf78-4497-82eb-76883882bd5d","Type":"ContainerStarted","Data":"23aa5e7303ffb528dc56c3d4c6566ad4a1b35f503b80ee812067bda2b8a331d6"} Jun 03 01:40:52.141719 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.141628 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" event={"ID":"1b755398-cf78-4497-82eb-76883882bd5d","Type":"ContainerStarted","Data":"6f553529c8ae4b50e7f65c18d2c1695e719a4274cd68a2136949d782d0de3c04"} Jun 03 01:40:52.146101 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.146043 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4jnwc" podStartSLOduration=2.510269772 podStartE2EDuration="22.146028295s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:30.933520769 +0000 UTC m=+1.588164033" lastFinishedPulling="2026-06-03 01:40:50.569279293 +0000 UTC m=+21.223922556" observedRunningTime="2026-06-03 01:40:51.159303861 +0000 UTC m=+21.813947156" watchObservedRunningTime="2026-06-03 01:40:52.146028295 +0000 UTC m=+22.800671579" Jun 03 01:40:52.162156 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.162036 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pjvzl" podStartSLOduration=2.55476896 podStartE2EDuration="22.1620167s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:30.939637676 +0000 UTC m=+1.594280949" lastFinishedPulling="2026-06-03 01:40:50.546885396 +0000 UTC m=+21.201528689" observedRunningTime="2026-06-03 01:40:52.161503305 +0000 UTC m=+22.816146597" watchObservedRunningTime="2026-06-03 01:40:52.1620167 +0000 UTC m=+22.816659983" Jun 03 01:40:52.178811 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.178716 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-gv8d4" podStartSLOduration=3.7237825940000002 podStartE2EDuration="23.178699259s" podCreationTimestamp="2026-06-03 01:40:29 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.114816787 +0000 UTC m=+1.769460045" lastFinishedPulling="2026-06-03 01:40:50.569733452 +0000 UTC m=+21.224376710" observedRunningTime="2026-06-03 01:40:52.17866139 +0000 UTC m=+22.833304670" watchObservedRunningTime="2026-06-03 01:40:52.178699259 +0000 UTC m=+22.833342541" Jun 03 01:40:52.229709 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.229666 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qwtj4" podStartSLOduration=2.849758548 podStartE2EDuration="22.229652122s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.144842891 +0000 UTC m=+1.799486149" lastFinishedPulling="2026-06-03 01:40:50.524736449 +0000 UTC m=+21.179379723" observedRunningTime="2026-06-03 01:40:52.229580107 +0000 UTC m=+22.884223486" watchObservedRunningTime="2026-06-03 01:40:52.229652122 +0000 UTC m=+22.884295402" Jun 03 01:40:52.229994 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.229966 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t55ss" podStartSLOduration=8.492213228 podStartE2EDuration="23.22995728s" podCreationTimestamp="2026-06-03 01:40:29 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.096037393 +0000 UTC m=+1.750680651" lastFinishedPulling="2026-06-03 01:40:45.833781445 +0000 UTC m=+16.488424703" observedRunningTime="2026-06-03 01:40:52.216723916 +0000 UTC m=+22.871367294" watchObservedRunningTime="2026-06-03 01:40:52.22995728 +0000 UTC m=+22.884600560" Jun 03 01:40:52.575867 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.575762 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:52.576405 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.576387 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:52.852129 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.852016 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T01:40:52.060823334Z","UUID":"c3767549-2907-4355-8783-3a18e3068a6e","Handler":null,"Name":"","Endpoint":""} Jun 03 01:40:52.854146 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.853922 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 01:40:52.854146 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.853954 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 01:40:52.935504 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.935469 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:52.935721 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:52.935518 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:52.935721 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:52.935611 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:52.935826 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:52.935729 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:53.146260 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:53.146227 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" event={"ID":"5760d5d3b69b8c00fa721bd2dc13a279","Type":"ContainerStarted","Data":"973e66ae59c58803e88fa75de611a1858e46e281ecfbcafdd7ea6b0aca77dea8"} Jun 03 01:40:53.146994 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:53.146976 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:53.147443 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:53.147422 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pjvzl" Jun 03 01:40:53.160908 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:53.160861 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-201.ec2.internal" podStartSLOduration=23.160844756 podStartE2EDuration="23.160844756s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:40:53.160623254 +0000 UTC m=+23.815266534" watchObservedRunningTime="2026-06-03 01:40:53.160844756 +0000 UTC m=+23.815488037" Jun 03 01:40:53.934828 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:53.934783 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:53.935002 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:53.934904 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:54.151767 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:54.151727 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"d5ad8c8af77efe13ba72971333962dd57cedbe846e44325bc7309935bce7467c"} Jun 03 01:40:54.153624 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:54.153593 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" event={"ID":"1b755398-cf78-4497-82eb-76883882bd5d","Type":"ContainerStarted","Data":"217fa90028c01202c89eeb20eeb0d64c8097e1b66fc510d97053e3be57050433"} Jun 03 01:40:54.172063 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:54.172014 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kwpfw" podStartSLOduration=2.252133214 podStartE2EDuration="24.171999265s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.163708737 +0000 UTC m=+1.818351997" lastFinishedPulling="2026-06-03 01:40:53.08357479 +0000 UTC m=+23.738218048" observedRunningTime="2026-06-03 01:40:54.171675218 +0000 UTC m=+24.826318498" watchObservedRunningTime="2026-06-03 01:40:54.171999265 +0000 UTC m=+24.826642598" Jun 03 01:40:54.935736 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:54.935458 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:54.935938 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:54.935493 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:54.935938 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:54.935910 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:54.936080 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:54.936053 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:55.785171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:55.785081 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:55.785644 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:55.785258 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:55.785644 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:55.785346 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret podName:b50f9317-b857-416c-b66a-1d7c38aa3c76 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:11.785324332 +0000 UTC m=+42.439967608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret") pod "global-pull-secret-syncer-49j2x" (UID: "b50f9317-b857-416c-b66a-1d7c38aa3c76") : object "kube-system"/"original-pull-secret" not registered Jun 03 01:40:55.935187 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:55.935148 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:55.935358 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:55.935288 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:56.935206 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:56.935174 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:56.935610 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:56.935293 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:56.935610 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:56.935359 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:56.935610 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:56.935486 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:57.162104 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:57.162070 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" event={"ID":"b8ae6056-f1d4-4ec0-8d77-3981347bcbd5","Type":"ContainerStarted","Data":"42b3593c8a574c28eb3e09a294a1f35a4080902f3588ec740ae48c0047ef822f"} Jun 03 01:40:57.162438 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:57.162415 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:57.162617 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:57.162444 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:57.178901 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:57.178829 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:57.190242 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:57.190197 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" podStartSLOduration=7.483455443 podStartE2EDuration="27.190185608s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.129284073 +0000 UTC m=+1.783927330" lastFinishedPulling="2026-06-03 01:40:50.836014237 +0000 UTC m=+21.490657495" observedRunningTime="2026-06-03 01:40:57.189920999 +0000 UTC m=+27.844564279" watchObservedRunningTime="2026-06-03 01:40:57.190185608 +0000 UTC m=+27.844828887" Jun 03 01:40:57.935473 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:57.935438 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:57.936104 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:57.935628 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:58.164913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.164875 2553 generic.go:358] "Generic (PLEG): container finished" podID="e1e6ff2f-3769-40c8-b42c-ba311fd1deca" containerID="649dc4f80a1ee09d742ca7965d98296cb6a6e63d2aeadb6b8afa431d58eda03c" exitCode=0 Jun 03 01:40:58.165064 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.164949 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerDied","Data":"649dc4f80a1ee09d742ca7965d98296cb6a6e63d2aeadb6b8afa431d58eda03c"} Jun 03 01:40:58.165498 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.165468 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:58.183903 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.183727 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:40:58.935088 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.935053 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:58.935254 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:58.935182 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:58.935254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.935233 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:58.935370 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:58.935337 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:40:58.968998 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.968964 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-66gkf"] Jun 03 01:40:58.971665 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.971641 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-49j2x"] Jun 03 01:40:58.971801 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.971759 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:40:58.971880 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:58.971862 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:40:58.972166 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:58.972147 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mv2xg"] Jun 03 01:40:59.168425 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:59.168391 2553 generic.go:358] "Generic (PLEG): container finished" podID="e1e6ff2f-3769-40c8-b42c-ba311fd1deca" containerID="77f79db9ded2cb55b8710f12ce4445bbc8f15ab7b2c5ec6779d00a672bb256bc" exitCode=0 Jun 03 01:40:59.168618 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:59.168507 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerDied","Data":"77f79db9ded2cb55b8710f12ce4445bbc8f15ab7b2c5ec6779d00a672bb256bc"} Jun 03 01:40:59.168618 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:59.168567 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:40:59.169001 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:40:59.168978 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:40:59.169117 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:59.169067 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:40:59.169470 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:40:59.169150 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:41:00.172110 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:00.172077 2553 generic.go:358] "Generic (PLEG): container finished" podID="e1e6ff2f-3769-40c8-b42c-ba311fd1deca" containerID="520c09fcbfd139b159adcd7206fc964873333ce73f36dbce815ad7fd22365e20" exitCode=0 Jun 03 01:41:00.172475 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:00.172162 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerDied","Data":"520c09fcbfd139b159adcd7206fc964873333ce73f36dbce815ad7fd22365e20"} Jun 03 01:41:00.935684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:00.935471 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:41:00.935861 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:00.935489 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:41:00.935861 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:00.935773 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:41:00.935861 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:00.935837 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:41:00.935861 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:00.935493 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:00.936024 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:00.935921 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:41:02.536597 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:02.536549 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:41:02.536997 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.536660 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:41:02.536997 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.536738 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:41:34.536719671 +0000 UTC m=+65.191362928 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 01:41:02.637418 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:02.637377 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:02.637617 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.637579 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 01:41:02.637617 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.637603 2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 01:41:02.637617 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.637616 2553 projected.go:194] Error preparing data for projected volume kube-api-access-nsckw for pod openshift-network-diagnostics/network-check-target-mv2xg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:41:02.637792 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.637686 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw podName:11abe95e-330a-4c61-8277-d49187327efc nodeName:}" failed. No retries permitted until 2026-06-03 01:41:34.637665986 +0000 UTC m=+65.292309263 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nsckw" (UniqueName: "kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw") pod "network-check-target-mv2xg" (UID: "11abe95e-330a-4c61-8277-d49187327efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 01:41:02.934839 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:02.934803 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:02.934839 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:02.934831 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:41:02.935052 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.934918 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv2xg" podUID="11abe95e-330a-4c61-8277-d49187327efc" Jun 03 01:41:02.935052 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:02.934953 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:41:02.935052 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.935024 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-66gkf" podUID="b11762dd-14ed-48d3-a346-a3997ab3567b" Jun 03 01:41:02.935149 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:02.935099 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-49j2x" podUID="b50f9317-b857-416c-b66a-1d7c38aa3c76" Jun 03 01:41:03.632610 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.632576 2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-201.ec2.internal" event="NodeReady" Jun 03 01:41:03.633082 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.632753 2553 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 01:41:03.666986 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.666949 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7787949c76-lxf6c"] Jun 03 01:41:03.669066 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.669045 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.672151 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.672127 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 01:41:03.672282 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.672173 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 01:41:03.672282 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.672175 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7lnrl\"" Jun 03 01:41:03.672397 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.672275 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 01:41:03.677450 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.677428 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 01:41:03.679448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.679424 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bwd6s"] Jun 03 01:41:03.681966 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.681942 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x4r47"] Jun 03 01:41:03.682121 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.682103 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.684034 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.684015 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7787949c76-lxf6c"] Jun 03 01:41:03.684149 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.684101 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:03.684581 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.684564 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 01:41:03.684665 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.684526 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2xcbn\"" Jun 03 01:41:03.684771 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.684757 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 01:41:03.686369 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.686352 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 01:41:03.686712 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.686694 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 01:41:03.686952 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.686936 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tsbnv\"" Jun 03 01:41:03.687173 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.687159 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 01:41:03.691881 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.691850 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x4r47"] Jun 03 01:41:03.696396 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.696370 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bwd6s"] Jun 03 01:41:03.746067 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746034 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a9af881-d17b-45be-a889-1f887e0ca248-ca-trust-extracted\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746067 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746075 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-trusted-ca\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746114 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-image-registry-private-configuration\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746156 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746181 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746207 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746230 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk7l9\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-kube-api-access-zk7l9\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746257 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c31246-3616-41a4-b95a-0a5242f1fa28-tmp-dir\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746283 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-installation-pull-secrets\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746324 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746322 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/74c31246-3616-41a4-b95a-0a5242f1fa28-config-volume\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.746664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746360 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28mv\" (UniqueName: \"kubernetes.io/projected/74c31246-3616-41a4-b95a-0a5242f1fa28-kube-api-access-l28mv\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.746664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746481 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-registry-certificates\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746522 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-bound-sa-token\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.746664 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.746564 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx2n7\" (UniqueName: \"kubernetes.io/projected/cc11711b-16fb-4148-bc98-81dbbc80b18d-kube-api-access-mx2n7\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:03.847553 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847492 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-registry-certificates\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847576 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-bound-sa-token\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847605 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mx2n7\" (UniqueName: \"kubernetes.io/projected/cc11711b-16fb-4148-bc98-81dbbc80b18d-kube-api-access-mx2n7\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:03.847713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847637 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a9af881-d17b-45be-a889-1f887e0ca248-ca-trust-extracted\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847658 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-trusted-ca\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847682 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-image-registry-private-configuration\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847703 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847721 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847740 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847756 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zk7l9\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-kube-api-access-zk7l9\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847776 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c31246-3616-41a4-b95a-0a5242f1fa28-tmp-dir\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847794 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-installation-pull-secrets\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847817 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/74c31246-3616-41a4-b95a-0a5242f1fa28-config-volume\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.847951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.847842 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l28mv\" (UniqueName: \"kubernetes.io/projected/74c31246-3616-41a4-b95a-0a5242f1fa28-kube-api-access-l28mv\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.848270 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.848116 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-registry-certificates\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.848325 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.848268 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:03.848325 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.848303 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:03.848325 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.848318 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:03.848325 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.848324 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:04.348308353 +0000 UTC m=+35.002951611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:03.848446 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.848357 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:04.348345113 +0000 UTC m=+35.002988371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:03.848686 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.848669 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c31246-3616-41a4-b95a-0a5242f1fa28-tmp-dir\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.849022 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.848996 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a9af881-d17b-45be-a889-1f887e0ca248-ca-trust-extracted\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.849257 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.849233 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:03.849328 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:03.849293 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:41:04.349276921 +0000 UTC m=+35.003920182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:03.849761 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.849725 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/74c31246-3616-41a4-b95a-0a5242f1fa28-config-volume\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.849849 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.849780 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-trusted-ca\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.853653 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.853629 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-installation-pull-secrets\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.853746 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.853687 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-image-registry-private-configuration\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.857304 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.857274 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-bound-sa-token\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:03.857401 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.857300 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28mv\" (UniqueName: \"kubernetes.io/projected/74c31246-3616-41a4-b95a-0a5242f1fa28-kube-api-access-l28mv\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:03.857401 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.857338 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx2n7\" (UniqueName: \"kubernetes.io/projected/cc11711b-16fb-4148-bc98-81dbbc80b18d-kube-api-access-mx2n7\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:03.857401 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:03.857283 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk7l9\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-kube-api-access-zk7l9\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:04.351507 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.351467 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:04.351720 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.351515 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:04.351720 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351635 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:04.351720 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351656 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:04.351720 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.351668 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:04.351720 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351696 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:05.351674484 +0000 UTC m=+36.006317743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:04.351924 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351742 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:04.351924 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351749 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:41:05.351713632 +0000 UTC m=+36.006356896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:04.351924 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351754 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:04.351924 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:04.351788 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:05.35177798 +0000 UTC m=+36.006421263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:04.934833 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.934791 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:04.935347 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.934844 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:41:04.935347 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.935019 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:41:04.937633 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.937606 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 01:41:04.937633 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.937617 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 01:41:04.938851 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.938831 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 01:41:04.938851 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.938848 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sx8gm\"" Jun 03 01:41:04.939030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.938907 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 01:41:04.939030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:04.938848 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hpsd7\"" Jun 03 01:41:05.360646 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:05.360602 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:05.360840 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.360733 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:05.360840 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.360760 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:05.360840 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.360823 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:07.36080246 +0000 UTC m=+38.015445724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:05.360840 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.360820 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:05.361056 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.360878 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:41:07.360862291 +0000 UTC m=+38.015505563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:05.361056 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:05.360737 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:05.361056 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:05.360922 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:05.361056 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.361013 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:05.361056 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:05.361048 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:07.361036677 +0000 UTC m=+38.015679949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:07.378966 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:07.378934 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:07.378966 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:07.378971 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:07.378993 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379092 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379099 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379170 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:41:11.379153858 +0000 UTC m=+42.033797116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379099 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379103 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379209 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:11.379199295 +0000 UTC m=+42.033842557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:07.379339 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:07.379255 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:11.379246703 +0000 UTC m=+42.033889989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:08.191339 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:08.191292 2553 generic.go:358] "Generic (PLEG): container finished" podID="e1e6ff2f-3769-40c8-b42c-ba311fd1deca" containerID="701e7a3d87de023db4caf8f750a2bc332d3945e4ea594d65d35fc1790ab44d66" exitCode=0 Jun 03 01:41:08.191494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:08.191346 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerDied","Data":"701e7a3d87de023db4caf8f750a2bc332d3945e4ea594d65d35fc1790ab44d66"} Jun 03 01:41:09.196065 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:09.196033 2553 generic.go:358] "Generic (PLEG): container finished" podID="e1e6ff2f-3769-40c8-b42c-ba311fd1deca" containerID="e57ba3cbb8b95a2f8edbd207feb7d4052a98c2a06637900cc9028a95032ddddd" exitCode=0 Jun 03 01:41:09.196431 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:09.196088 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerDied","Data":"e57ba3cbb8b95a2f8edbd207feb7d4052a98c2a06637900cc9028a95032ddddd"} Jun 03 01:41:10.203703 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:10.203663 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" event={"ID":"e1e6ff2f-3769-40c8-b42c-ba311fd1deca","Type":"ContainerStarted","Data":"4b4d5c6e0c77439866eb0c00e56a9ffc82e07da6210f6ce7e320dddd3e1c5c9c"} Jun 03 01:41:10.227971 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:10.227887 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rd2nk" podStartSLOduration=4.332277947 podStartE2EDuration="40.227872816s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:40:31.181922798 +0000 UTC m=+1.836566056" lastFinishedPulling="2026-06-03 01:41:07.077517665 +0000 UTC m=+37.732160925" observedRunningTime="2026-06-03 01:41:10.226183254 +0000 UTC m=+40.880826532" watchObservedRunningTime="2026-06-03 01:41:10.227872816 +0000 UTC m=+40.882516095" Jun 03 01:41:11.414985 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:11.414941 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:11.414985 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:11.414984 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:11.415004 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415101 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415101 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415104 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415159 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:19.415144535 +0000 UTC m=+50.069787793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415159 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415174 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:41:19.415166508 +0000 UTC m=+50.069809766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:11.415386 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:11.415193 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:19.415179118 +0000 UTC m=+50.069822376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:11.817586 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:11.817477 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:41:11.820959 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:11.820928 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50f9317-b857-416c-b66a-1d7c38aa3c76-original-pull-secret\") pod \"global-pull-secret-syncer-49j2x\" (UID: \"b50f9317-b857-416c-b66a-1d7c38aa3c76\") " pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:41:11.854214 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:11.854171 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-49j2x" Jun 03 01:41:12.013967 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:12.013932 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-49j2x"] Jun 03 01:41:12.017455 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:41:12.017422 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50f9317_b857_416c_b66a_1d7c38aa3c76.slice/crio-1aabd68e5c6dc0b79f6564e552fafd11a43632ab01b880632f0ed81acd7d5393 WatchSource:0}: Error finding container 1aabd68e5c6dc0b79f6564e552fafd11a43632ab01b880632f0ed81acd7d5393: Status 404 returned error can't find the container with id 1aabd68e5c6dc0b79f6564e552fafd11a43632ab01b880632f0ed81acd7d5393 Jun 03 01:41:12.208210 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:12.208171 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-49j2x" event={"ID":"b50f9317-b857-416c-b66a-1d7c38aa3c76","Type":"ContainerStarted","Data":"1aabd68e5c6dc0b79f6564e552fafd11a43632ab01b880632f0ed81acd7d5393"} Jun 03 01:41:18.220328 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:18.220293 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-49j2x" event={"ID":"b50f9317-b857-416c-b66a-1d7c38aa3c76","Type":"ContainerStarted","Data":"34d42a11d39e9d0ab961d1e3d3e39ba4eca8c8a9cf3fff976176317653a73898"} Jun 03 01:41:18.234351 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:18.234298 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-49j2x" podStartSLOduration=34.08189001 podStartE2EDuration="39.234283585s" podCreationTimestamp="2026-06-03 01:40:39 +0000 UTC" firstStartedPulling="2026-06-03 01:41:12.019311391 +0000 UTC m=+42.673954664" lastFinishedPulling="2026-06-03 01:41:17.171704969 +0000 UTC m=+47.826348239" observedRunningTime="2026-06-03 01:41:18.233573289 +0000 UTC m=+48.888216570" watchObservedRunningTime="2026-06-03 01:41:18.234283585 +0000 UTC m=+48.888926844" Jun 03 01:41:19.478099 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:19.478060 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:19.478099 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:19.478101 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:19.478132 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478207 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478218 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478270 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:41:35.478256021 +0000 UTC m=+66.132899279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478278 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478292 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478285 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:35.478279649 +0000 UTC m=+66.132922906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:19.478560 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:19.478355 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:41:35.478330498 +0000 UTC m=+66.132973768 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:30.183213 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:30.183184 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-475tz" Jun 03 01:41:34.585779 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.585716 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:41:34.588583 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.588564 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 01:41:34.596697 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:34.596669 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 01:41:34.596766 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:34.596754 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs podName:b11762dd-14ed-48d3-a346-a3997ab3567b nodeName:}" failed. No retries permitted until 2026-06-03 01:42:38.596732599 +0000 UTC m=+129.251376087 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs") pod "network-metrics-daemon-66gkf" (UID: "b11762dd-14ed-48d3-a346-a3997ab3567b") : secret "metrics-daemon-secret" not found Jun 03 01:41:34.686574 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.686524 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:34.689942 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.689924 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 01:41:34.699628 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.699605 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 01:41:34.710259 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.710230 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsckw\" (UniqueName: \"kubernetes.io/projected/11abe95e-330a-4c61-8277-d49187327efc-kube-api-access-nsckw\") pod \"network-check-target-mv2xg\" (UID: \"11abe95e-330a-4c61-8277-d49187327efc\") " pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:34.950990 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.950902 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hpsd7\"" Jun 03 01:41:34.959043 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:34.959017 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:35.079721 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:35.079685 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mv2xg"] Jun 03 01:41:35.082663 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:41:35.082631 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11abe95e_330a_4c61_8277_d49187327efc.slice/crio-f9d8070c839d50b5b212352b360d99312b02be084c07bc70d3047a1a8cb331dd WatchSource:0}: Error finding container f9d8070c839d50b5b212352b360d99312b02be084c07bc70d3047a1a8cb331dd: Status 404 returned error can't find the container with id f9d8070c839d50b5b212352b360d99312b02be084c07bc70d3047a1a8cb331dd Jun 03 01:41:35.254214 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:35.254124 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mv2xg" event={"ID":"11abe95e-330a-4c61-8277-d49187327efc","Type":"ContainerStarted","Data":"f9d8070c839d50b5b212352b360d99312b02be084c07bc70d3047a1a8cb331dd"} Jun 03 01:41:35.491714 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:35.491679 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:41:35.491895 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:35.491723 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:41:35.491895 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:35.491752 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:41:35.491895 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.491846 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:41:35.491895 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.491887 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:41:35.492079 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.491901 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:41:35.492079 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.491847 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:41:35.492079 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.491942 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:42:07.49191616 +0000 UTC m=+98.146559423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:41:35.492079 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.491986 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:42:07.491969691 +0000 UTC m=+98.146612957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:41:35.492079 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:41:35.492009 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:42:07.491999852 +0000 UTC m=+98.146643125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:41:38.261150 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:38.261110 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mv2xg" event={"ID":"11abe95e-330a-4c61-8277-d49187327efc","Type":"ContainerStarted","Data":"47cace496a3a6c97b5829ca9cd0fc7319b79c3840584c788e0356e433ee89aad"} Jun 03 01:41:38.261576 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:38.261247 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:41:38.276385 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:41:38.276337 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mv2xg" podStartSLOduration=66.302671733 podStartE2EDuration="1m9.276323366s" podCreationTimestamp="2026-06-03 01:40:29 +0000 UTC" firstStartedPulling="2026-06-03 01:41:35.08494089 +0000 UTC m=+65.739584148" lastFinishedPulling="2026-06-03 01:41:38.058592509 +0000 UTC m=+68.713235781" observedRunningTime="2026-06-03 01:41:38.275737172 +0000 UTC m=+68.930380451" watchObservedRunningTime="2026-06-03 01:41:38.276323366 +0000 UTC m=+68.930966623" Jun 03 01:42:07.526514 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:07.526469 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:42:07.526514 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:07.526513 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:07.526561 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") pod \"image-registry-7787949c76-lxf6c\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526645 2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526653 2553 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526673 2553 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7787949c76-lxf6c: secret "image-registry-tls" not found Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526711 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert podName:cc11711b-16fb-4148-bc98-81dbbc80b18d nodeName:}" failed. No retries permitted until 2026-06-03 01:43:11.526691781 +0000 UTC m=+162.181335038 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert") pod "ingress-canary-x4r47" (UID: "cc11711b-16fb-4148-bc98-81dbbc80b18d") : secret "canary-serving-cert" not found Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526732 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls podName:4a9af881-d17b-45be-a889-1f887e0ca248 nodeName:}" failed. No retries permitted until 2026-06-03 01:43:11.526722554 +0000 UTC m=+162.181365833 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls") pod "image-registry-7787949c76-lxf6c" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248") : secret "image-registry-tls" not found Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526645 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 01:42:07.527060 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:07.526800 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls podName:74c31246-3616-41a4-b95a-0a5242f1fa28 nodeName:}" failed. No retries permitted until 2026-06-03 01:43:11.526786669 +0000 UTC m=+162.181429926 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls") pod "dns-default-bwd6s" (UID: "74c31246-3616-41a4-b95a-0a5242f1fa28") : secret "dns-default-metrics-tls" not found Jun 03 01:42:07.970713 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:07.970685 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gv8d4_ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4/dns-node-resolver/0.log" Jun 03 01:42:08.970961 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:08.970932 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t55ss_c3da5575-9d26-4064-85da-7c11be51c8d4/node-ca/0.log" Jun 03 01:42:09.265745 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:09.265666 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mv2xg" Jun 03 01:42:12.963909 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.963875 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr"] Jun 03 01:42:12.965665 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.965641 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-xxmhj"] Jun 03 01:42:12.965799 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.965779 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" Jun 03 01:42:12.967138 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.967118 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7"] Jun 03 01:42:12.967284 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.967266 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:12.968218 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.968199 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 03 01:42:12.968319 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.968205 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 03 01:42:12.968367 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.968204 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8nvl2\"" Jun 03 01:42:12.968741 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.968724 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:12.969741 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.969722 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 03 01:42:12.969809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.969723 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8rkpc\"" Jun 03 01:42:12.970071 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.970049 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 03 01:42:12.970186 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.970069 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 03 01:42:12.970186 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.970049 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 03 01:42:12.971101 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.971082 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 03 01:42:12.971101 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.971105 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 03 01:42:12.971300 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.971280 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 03 01:42:12.971397 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.971376 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-25pdx\"" Jun 03 01:42:12.971914 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.971894 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 03 01:42:12.974750 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.974731 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 03 01:42:12.977355 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.977335 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr"] Jun 03 01:42:12.978270 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.978238 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7"] Jun 03 01:42:12.978930 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:12.978912 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-xxmhj"] Jun 03 01:42:13.067983 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.067935 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b16cd5f5-9216-4743-a5ab-97375ec370c1-trusted-ca\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.068152 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068005 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f7tx\" (UniqueName: \"kubernetes.io/projected/fe9034a5-22c3-4644-9035-492d013fa9c0-kube-api-access-5f7tx\") pod \"volume-data-source-validator-7598fd5bd-c49dr\" (UID: \"fe9034a5-22c3-4644-9035-492d013fa9c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" Jun 03 01:42:13.068152 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068043 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b16cd5f5-9216-4743-a5ab-97375ec370c1-config\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.068152 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068065 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm97k\" (UniqueName: \"kubernetes.io/projected/b16cd5f5-9216-4743-a5ab-97375ec370c1-kube-api-access-bm97k\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.068152 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068086 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.068152 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068137 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv5pt\" (UniqueName: \"kubernetes.io/projected/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-kube-api-access-qv5pt\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.068359 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068231 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b16cd5f5-9216-4743-a5ab-97375ec370c1-serving-cert\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.068359 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.068277 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.169273 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169236 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bm97k\" (UniqueName: \"kubernetes.io/projected/b16cd5f5-9216-4743-a5ab-97375ec370c1-kube-api-access-bm97k\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.169273 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169276 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.169447 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169390 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qv5pt\" (UniqueName: \"kubernetes.io/projected/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-kube-api-access-qv5pt\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.169447 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169427 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b16cd5f5-9216-4743-a5ab-97375ec370c1-serving-cert\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.169513 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169492 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.169652 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169633 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b16cd5f5-9216-4743-a5ab-97375ec370c1-trusted-ca\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.169719 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169690 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5f7tx\" (UniqueName: \"kubernetes.io/projected/fe9034a5-22c3-4644-9035-492d013fa9c0-kube-api-access-5f7tx\") pod \"volume-data-source-validator-7598fd5bd-c49dr\" (UID: \"fe9034a5-22c3-4644-9035-492d013fa9c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" Jun 03 01:42:13.169777 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169730 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b16cd5f5-9216-4743-a5ab-97375ec370c1-config\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.169874 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.169855 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.170311 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.170249 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b16cd5f5-9216-4743-a5ab-97375ec370c1-config\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.170524 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.170507 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b16cd5f5-9216-4743-a5ab-97375ec370c1-trusted-ca\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.171929 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.171908 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b16cd5f5-9216-4743-a5ab-97375ec370c1-serving-cert\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.172037 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.171950 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.178345 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.178319 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f7tx\" (UniqueName: \"kubernetes.io/projected/fe9034a5-22c3-4644-9035-492d013fa9c0-kube-api-access-5f7tx\") pod \"volume-data-source-validator-7598fd5bd-c49dr\" (UID: \"fe9034a5-22c3-4644-9035-492d013fa9c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" Jun 03 01:42:13.179108 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.179089 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv5pt\" (UniqueName: \"kubernetes.io/projected/8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0-kube-api-access-qv5pt\") pod \"kube-storage-version-migrator-operator-96ddc787c-tqsx7\" (UID: \"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.179222 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.179199 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm97k\" (UniqueName: \"kubernetes.io/projected/b16cd5f5-9216-4743-a5ab-97375ec370c1-kube-api-access-bm97k\") pod \"console-operator-86d7c6cc77-xxmhj\" (UID: \"b16cd5f5-9216-4743-a5ab-97375ec370c1\") " pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.277733 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.277632 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" Jun 03 01:42:13.285627 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.285600 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:13.290384 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.290356 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" Jun 03 01:42:13.423812 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.423777 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr"] Jun 03 01:42:13.427417 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:13.427384 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe9034a5_22c3_4644_9035_492d013fa9c0.slice/crio-9b98164af9b8f0163ea1452a0b43900bba17faff3b4f7aa549522a928659566b WatchSource:0}: Error finding container 9b98164af9b8f0163ea1452a0b43900bba17faff3b4f7aa549522a928659566b: Status 404 returned error can't find the container with id 9b98164af9b8f0163ea1452a0b43900bba17faff3b4f7aa549522a928659566b Jun 03 01:42:13.435624 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.435598 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-xxmhj"] Jun 03 01:42:13.438610 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:13.438575 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16cd5f5_9216_4743_a5ab_97375ec370c1.slice/crio-5651f1645e82d70ed07772d03532bd91ab8652c2b953ad2eef08530a0db745a8 WatchSource:0}: Error finding container 5651f1645e82d70ed07772d03532bd91ab8652c2b953ad2eef08530a0db745a8: Status 404 returned error can't find the container with id 5651f1645e82d70ed07772d03532bd91ab8652c2b953ad2eef08530a0db745a8 Jun 03 01:42:13.455454 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:13.455426 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7"] Jun 03 01:42:13.458476 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:13.458451 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ffbed46_5345_4dfc_8bf6_b2ee67fa59b0.slice/crio-67411a1886bb46aabe867707895ffa9238e93c7f6bf564de6f7379a05a993c63 WatchSource:0}: Error finding container 67411a1886bb46aabe867707895ffa9238e93c7f6bf564de6f7379a05a993c63: Status 404 returned error can't find the container with id 67411a1886bb46aabe867707895ffa9238e93c7f6bf564de6f7379a05a993c63 Jun 03 01:42:14.331126 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:14.331067 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" event={"ID":"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0","Type":"ContainerStarted","Data":"67411a1886bb46aabe867707895ffa9238e93c7f6bf564de6f7379a05a993c63"} Jun 03 01:42:14.332447 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:14.332409 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" event={"ID":"b16cd5f5-9216-4743-a5ab-97375ec370c1","Type":"ContainerStarted","Data":"5651f1645e82d70ed07772d03532bd91ab8652c2b953ad2eef08530a0db745a8"} Jun 03 01:42:14.333626 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:14.333594 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" event={"ID":"fe9034a5-22c3-4644-9035-492d013fa9c0","Type":"ContainerStarted","Data":"9b98164af9b8f0163ea1452a0b43900bba17faff3b4f7aa549522a928659566b"} Jun 03 01:42:16.339552 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.339505 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" event={"ID":"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0","Type":"ContainerStarted","Data":"19cb3ce941d3c0f577d05c24ec94bde992062d8e2562bd6158505cf8b095d2e7"} Jun 03 01:42:16.341074 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.341052 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/0.log" Jun 03 01:42:16.341169 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.341096 2553 generic.go:358] "Generic (PLEG): container finished" podID="b16cd5f5-9216-4743-a5ab-97375ec370c1" containerID="4153b4d1cf4aef1ef3b3344e0ba06d166307d1339ee5b674abca6e5f3f2c5770" exitCode=255 Jun 03 01:42:16.341169 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.341162 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" event={"ID":"b16cd5f5-9216-4743-a5ab-97375ec370c1","Type":"ContainerDied","Data":"4153b4d1cf4aef1ef3b3344e0ba06d166307d1339ee5b674abca6e5f3f2c5770"} Jun 03 01:42:16.341342 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.341327 2553 scope.go:117] "RemoveContainer" containerID="4153b4d1cf4aef1ef3b3344e0ba06d166307d1339ee5b674abca6e5f3f2c5770" Jun 03 01:42:16.342497 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.342478 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" event={"ID":"fe9034a5-22c3-4644-9035-492d013fa9c0","Type":"ContainerStarted","Data":"df89c066ef1a11ea78f6a6622a5ef564296c7427a7ef4b8ee6de55bc2be53ef2"} Jun 03 01:42:16.354323 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.354269 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" podStartSLOduration=1.866051781 podStartE2EDuration="4.354253568s" podCreationTimestamp="2026-06-03 01:42:12 +0000 UTC" firstStartedPulling="2026-06-03 01:42:13.460218555 +0000 UTC m=+104.114861813" lastFinishedPulling="2026-06-03 01:42:15.948420337 +0000 UTC m=+106.603063600" observedRunningTime="2026-06-03 01:42:16.353880743 +0000 UTC m=+107.008524024" watchObservedRunningTime="2026-06-03 01:42:16.354253568 +0000 UTC m=+107.008896849" Jun 03 01:42:16.367332 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:16.367283 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-c49dr" podStartSLOduration=1.85256999 podStartE2EDuration="4.367268279s" podCreationTimestamp="2026-06-03 01:42:12 +0000 UTC" firstStartedPulling="2026-06-03 01:42:13.429272109 +0000 UTC m=+104.083915367" lastFinishedPulling="2026-06-03 01:42:15.943970398 +0000 UTC m=+106.598613656" observedRunningTime="2026-06-03 01:42:16.366734977 +0000 UTC m=+107.021378258" watchObservedRunningTime="2026-06-03 01:42:16.367268279 +0000 UTC m=+107.021911558" Jun 03 01:42:17.346548 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.346503 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/1.log" Jun 03 01:42:17.346971 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.346896 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/0.log" Jun 03 01:42:17.346971 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.346931 2553 generic.go:358] "Generic (PLEG): container finished" podID="b16cd5f5-9216-4743-a5ab-97375ec370c1" containerID="2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586" exitCode=255 Jun 03 01:42:17.347079 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.347027 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" event={"ID":"b16cd5f5-9216-4743-a5ab-97375ec370c1","Type":"ContainerDied","Data":"2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586"} Jun 03 01:42:17.347079 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.347075 2553 scope.go:117] "RemoveContainer" containerID="4153b4d1cf4aef1ef3b3344e0ba06d166307d1339ee5b674abca6e5f3f2c5770" Jun 03 01:42:17.347349 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.347326 2553 scope.go:117] "RemoveContainer" containerID="2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586" Jun 03 01:42:17.347571 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:17.347548 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-xxmhj_openshift-console-operator(b16cd5f5-9216-4743-a5ab-97375ec370c1)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" podUID="b16cd5f5-9216-4743-a5ab-97375ec370c1" Jun 03 01:42:17.996218 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.996176 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt"] Jun 03 01:42:17.999055 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:17.999032 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" Jun 03 01:42:18.001483 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.001463 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r2ngt\"" Jun 03 01:42:18.001611 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.001501 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 03 01:42:18.002516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.002488 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 03 01:42:18.008021 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.007997 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt"] Jun 03 01:42:18.110399 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.110364 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv7jb\" (UniqueName: \"kubernetes.io/projected/6ea6b0ba-a9a8-4710-a542-79a5aace89d3-kube-api-access-cv7jb\") pod \"migrator-96c6f749-cx5mt\" (UID: \"6ea6b0ba-a9a8-4710-a542-79a5aace89d3\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" Jun 03 01:42:18.211458 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.211417 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7jb\" (UniqueName: \"kubernetes.io/projected/6ea6b0ba-a9a8-4710-a542-79a5aace89d3-kube-api-access-cv7jb\") pod \"migrator-96c6f749-cx5mt\" (UID: \"6ea6b0ba-a9a8-4710-a542-79a5aace89d3\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" Jun 03 01:42:18.219217 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.219188 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv7jb\" (UniqueName: \"kubernetes.io/projected/6ea6b0ba-a9a8-4710-a542-79a5aace89d3-kube-api-access-cv7jb\") pod \"migrator-96c6f749-cx5mt\" (UID: \"6ea6b0ba-a9a8-4710-a542-79a5aace89d3\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" Jun 03 01:42:18.308498 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.308413 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" Jun 03 01:42:18.350355 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.350327 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/1.log" Jun 03 01:42:18.350774 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.350683 2553 scope.go:117] "RemoveContainer" containerID="2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586" Jun 03 01:42:18.350880 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:18.350862 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-xxmhj_openshift-console-operator(b16cd5f5-9216-4743-a5ab-97375ec370c1)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" podUID="b16cd5f5-9216-4743-a5ab-97375ec370c1" Jun 03 01:42:18.434441 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:18.434393 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt"] Jun 03 01:42:18.439293 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:18.439263 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea6b0ba_a9a8_4710_a542_79a5aace89d3.slice/crio-60a354753a820530cc87e11b45537080eafd47d1b7bc7dc50ee4ba835cb5c62e WatchSource:0}: Error finding container 60a354753a820530cc87e11b45537080eafd47d1b7bc7dc50ee4ba835cb5c62e: Status 404 returned error can't find the container with id 60a354753a820530cc87e11b45537080eafd47d1b7bc7dc50ee4ba835cb5c62e Jun 03 01:42:19.353378 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.353345 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" event={"ID":"6ea6b0ba-a9a8-4710-a542-79a5aace89d3","Type":"ContainerStarted","Data":"60a354753a820530cc87e11b45537080eafd47d1b7bc7dc50ee4ba835cb5c62e"} Jun 03 01:42:19.414981 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.414944 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wvhlg"] Jun 03 01:42:19.417095 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.417078 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.419721 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.419693 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jv5mr\"" Jun 03 01:42:19.419721 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.419713 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 01:42:19.419940 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.419734 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 01:42:19.419940 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.419694 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 01:42:19.420829 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.420804 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 01:42:19.426102 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.426082 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvhlg"] Jun 03 01:42:19.522736 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.522683 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snnh9\" (UniqueName: \"kubernetes.io/projected/57f1efd1-c82a-448a-b70a-081c94dc6f71-kube-api-access-snnh9\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.522736 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.522741 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57f1efd1-c82a-448a-b70a-081c94dc6f71-crio-socket\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.522976 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.522793 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f1efd1-c82a-448a-b70a-081c94dc6f71-data-volume\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.522976 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.522860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.522976 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.522911 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57f1efd1-c82a-448a-b70a-081c94dc6f71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.623976 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.623891 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57f1efd1-c82a-448a-b70a-081c94dc6f71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.623984 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-snnh9\" (UniqueName: \"kubernetes.io/projected/57f1efd1-c82a-448a-b70a-081c94dc6f71-kube-api-access-snnh9\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.624005 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57f1efd1-c82a-448a-b70a-081c94dc6f71-crio-socket\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.624025 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f1efd1-c82a-448a-b70a-081c94dc6f71-data-volume\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.624048 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624317 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.624120 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57f1efd1-c82a-448a-b70a-081c94dc6f71-crio-socket\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624317 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:19.624155 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 03 01:42:19.624317 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:19.624226 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls podName:57f1efd1-c82a-448a-b70a-081c94dc6f71 nodeName:}" failed. No retries permitted until 2026-06-03 01:42:20.124207699 +0000 UTC m=+110.778850961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvhlg" (UID: "57f1efd1-c82a-448a-b70a-081c94dc6f71") : secret "insights-runtime-extractor-tls" not found Jun 03 01:42:19.624456 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.624383 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57f1efd1-c82a-448a-b70a-081c94dc6f71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.624456 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.624389 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f1efd1-c82a-448a-b70a-081c94dc6f71-data-volume\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:19.634922 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:19.634887 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snnh9\" (UniqueName: \"kubernetes.io/projected/57f1efd1-c82a-448a-b70a-081c94dc6f71-kube-api-access-snnh9\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:20.127726 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:20.127664 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:20.127897 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:20.127831 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 03 01:42:20.127970 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:20.127902 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls podName:57f1efd1-c82a-448a-b70a-081c94dc6f71 nodeName:}" failed. No retries permitted until 2026-06-03 01:42:21.127885808 +0000 UTC m=+111.782529066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvhlg" (UID: "57f1efd1-c82a-448a-b70a-081c94dc6f71") : secret "insights-runtime-extractor-tls" not found Jun 03 01:42:20.356769 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:20.356721 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" event={"ID":"6ea6b0ba-a9a8-4710-a542-79a5aace89d3","Type":"ContainerStarted","Data":"7fa4e31389afddcdf394e75d01de2a520166d01ab34a539d0eba4866afb9da27"} Jun 03 01:42:20.356769 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:20.356772 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" event={"ID":"6ea6b0ba-a9a8-4710-a542-79a5aace89d3","Type":"ContainerStarted","Data":"73976819f2610b7e99fce5a213b794ca7f36cfbe1656f087e11f3098c408b80b"} Jun 03 01:42:20.373658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:20.373608 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-cx5mt" podStartSLOduration=1.9163912870000002 podStartE2EDuration="3.373592769s" podCreationTimestamp="2026-06-03 01:42:17 +0000 UTC" firstStartedPulling="2026-06-03 01:42:18.441597722 +0000 UTC m=+109.096240980" lastFinishedPulling="2026-06-03 01:42:19.898799201 +0000 UTC m=+110.553442462" observedRunningTime="2026-06-03 01:42:20.372676115 +0000 UTC m=+111.027319396" watchObservedRunningTime="2026-06-03 01:42:20.373592769 +0000 UTC m=+111.028236049" Jun 03 01:42:21.135688 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:21.135649 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:21.135866 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:21.135801 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 03 01:42:21.135908 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:21.135873 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls podName:57f1efd1-c82a-448a-b70a-081c94dc6f71 nodeName:}" failed. No retries permitted until 2026-06-03 01:42:23.13585695 +0000 UTC m=+113.790500209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvhlg" (UID: "57f1efd1-c82a-448a-b70a-081c94dc6f71") : secret "insights-runtime-extractor-tls" not found Jun 03 01:42:23.151104 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:23.151067 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:23.151483 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:23.151201 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 03 01:42:23.151483 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:23.151267 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls podName:57f1efd1-c82a-448a-b70a-081c94dc6f71 nodeName:}" failed. No retries permitted until 2026-06-03 01:42:27.151252179 +0000 UTC m=+117.805895437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvhlg" (UID: "57f1efd1-c82a-448a-b70a-081c94dc6f71") : secret "insights-runtime-extractor-tls" not found Jun 03 01:42:23.286107 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:23.286063 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:23.286107 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:23.286103 2553 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:23.286500 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:23.286483 2553 scope.go:117] "RemoveContainer" containerID="2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586" Jun 03 01:42:23.286686 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:23.286669 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-xxmhj_openshift-console-operator(b16cd5f5-9216-4743-a5ab-97375ec370c1)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" podUID="b16cd5f5-9216-4743-a5ab-97375ec370c1" Jun 03 01:42:27.185184 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:27.185147 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:27.187657 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:27.187632 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57f1efd1-c82a-448a-b70a-081c94dc6f71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvhlg\" (UID: \"57f1efd1-c82a-448a-b70a-081c94dc6f71\") " pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:27.226926 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:27.226882 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvhlg" Jun 03 01:42:27.353043 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:27.353010 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvhlg"] Jun 03 01:42:27.356434 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:27.356404 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f1efd1_c82a_448a_b70a_081c94dc6f71.slice/crio-17b988301d6f28d3afb1f8779cd9d4b6ff472b6ec2c42d2c1915f083a79b5e3f WatchSource:0}: Error finding container 17b988301d6f28d3afb1f8779cd9d4b6ff472b6ec2c42d2c1915f083a79b5e3f: Status 404 returned error can't find the container with id 17b988301d6f28d3afb1f8779cd9d4b6ff472b6ec2c42d2c1915f083a79b5e3f Jun 03 01:42:27.375873 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:27.375846 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvhlg" event={"ID":"57f1efd1-c82a-448a-b70a-081c94dc6f71","Type":"ContainerStarted","Data":"17b988301d6f28d3afb1f8779cd9d4b6ff472b6ec2c42d2c1915f083a79b5e3f"} Jun 03 01:42:28.380087 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:28.379999 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvhlg" event={"ID":"57f1efd1-c82a-448a-b70a-081c94dc6f71","Type":"ContainerStarted","Data":"c4b05d79b986e4ae128cd95ee258275f8e1c0ad4994bbc255de008097f6eb611"} Jun 03 01:42:28.380087 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:28.380040 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvhlg" event={"ID":"57f1efd1-c82a-448a-b70a-081c94dc6f71","Type":"ContainerStarted","Data":"2fdbe70cd2170ebd22b90710386d02106262c8b5b9b36f90c4d14a6e7d183f16"} Jun 03 01:42:30.387069 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:30.386982 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvhlg" event={"ID":"57f1efd1-c82a-448a-b70a-081c94dc6f71","Type":"ContainerStarted","Data":"e5e4e3b383d6a61d6893d14c858fa51e6546a06974689924eae4ab138397065c"} Jun 03 01:42:30.406086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:30.406038 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wvhlg" podStartSLOduration=8.734369000000001 podStartE2EDuration="11.406023953s" podCreationTimestamp="2026-06-03 01:42:19 +0000 UTC" firstStartedPulling="2026-06-03 01:42:27.424356483 +0000 UTC m=+118.078999741" lastFinishedPulling="2026-06-03 01:42:30.096011422 +0000 UTC m=+120.750654694" observedRunningTime="2026-06-03 01:42:30.405677532 +0000 UTC m=+121.060320814" watchObservedRunningTime="2026-06-03 01:42:30.406023953 +0000 UTC m=+121.060667232" Jun 03 01:42:35.935851 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:35.935815 2553 scope.go:117] "RemoveContainer" containerID="2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586" Jun 03 01:42:36.402568 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:36.402519 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:42:36.402929 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:36.402914 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/1.log" Jun 03 01:42:36.402980 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:36.402949 2553 generic.go:358] "Generic (PLEG): container finished" podID="b16cd5f5-9216-4743-a5ab-97375ec370c1" containerID="d18160cfba22431090f928602f9ab1719afe4c9518e08a8de0137b9b7b5dc271" exitCode=255 Jun 03 01:42:36.403017 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:36.403003 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" event={"ID":"b16cd5f5-9216-4743-a5ab-97375ec370c1","Type":"ContainerDied","Data":"d18160cfba22431090f928602f9ab1719afe4c9518e08a8de0137b9b7b5dc271"} Jun 03 01:42:36.403068 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:36.403037 2553 scope.go:117] "RemoveContainer" containerID="2d2f6753bfd95463c3b40e37e5e11bb45d615105267067e6a460491e7c54a586" Jun 03 01:42:36.403369 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:36.403350 2553 scope.go:117] "RemoveContainer" containerID="d18160cfba22431090f928602f9ab1719afe4c9518e08a8de0137b9b7b5dc271" Jun 03 01:42:36.403578 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:36.403527 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-xxmhj_openshift-console-operator(b16cd5f5-9216-4743-a5ab-97375ec370c1)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" podUID="b16cd5f5-9216-4743-a5ab-97375ec370c1" Jun 03 01:42:37.406644 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:37.406612 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:42:38.672199 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:38.672158 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:42:38.674601 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:38.674581 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11762dd-14ed-48d3-a346-a3997ab3567b-metrics-certs\") pod \"network-metrics-daemon-66gkf\" (UID: \"b11762dd-14ed-48d3-a346-a3997ab3567b\") " pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:42:38.862937 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:38.862905 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sx8gm\"" Jun 03 01:42:38.870891 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:38.870872 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-66gkf" Jun 03 01:42:39.009034 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:39.009002 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-66gkf"] Jun 03 01:42:39.012032 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:39.012007 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11762dd_14ed_48d3_a346_a3997ab3567b.slice/crio-90aaba65b8dd6ae8337d7bd4295ef285dceeca0dc86ec16cd990a78da04ec387 WatchSource:0}: Error finding container 90aaba65b8dd6ae8337d7bd4295ef285dceeca0dc86ec16cd990a78da04ec387: Status 404 returned error can't find the container with id 90aaba65b8dd6ae8337d7bd4295ef285dceeca0dc86ec16cd990a78da04ec387 Jun 03 01:42:39.411617 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:39.411580 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-66gkf" event={"ID":"b11762dd-14ed-48d3-a346-a3997ab3567b","Type":"ContainerStarted","Data":"90aaba65b8dd6ae8337d7bd4295ef285dceeca0dc86ec16cd990a78da04ec387"} Jun 03 01:42:40.415617 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:40.415584 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-66gkf" event={"ID":"b11762dd-14ed-48d3-a346-a3997ab3567b","Type":"ContainerStarted","Data":"3067e42a1782396358b8b8dbaa9f59faca80a25fa824d2582f52186bf7ad7620"} Jun 03 01:42:40.415617 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:40.415619 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-66gkf" event={"ID":"b11762dd-14ed-48d3-a346-a3997ab3567b","Type":"ContainerStarted","Data":"8dfe4b56c671f5c225d518819db32648952a18311bce36743e9e0808de878153"} Jun 03 01:42:40.431787 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:40.431738 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-66gkf" podStartSLOduration=129.261015615 podStartE2EDuration="2m10.431722904s" podCreationTimestamp="2026-06-03 01:40:30 +0000 UTC" firstStartedPulling="2026-06-03 01:42:39.0139153 +0000 UTC m=+129.668558561" lastFinishedPulling="2026-06-03 01:42:40.184622589 +0000 UTC m=+130.839265850" observedRunningTime="2026-06-03 01:42:40.429919095 +0000 UTC m=+131.084562375" watchObservedRunningTime="2026-06-03 01:42:40.431722904 +0000 UTC m=+131.086366183" Jun 03 01:42:41.950642 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.950606 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv"] Jun 03 01:42:41.952727 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.952711 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh"] Jun 03 01:42:41.952880 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.952864 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:41.954506 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.954488 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" Jun 03 01:42:41.955711 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.955695 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 03 01:42:41.956676 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.956659 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sv6l9\"" Jun 03 01:42:41.957229 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.957215 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7bx4m\"" Jun 03 01:42:41.959580 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.959554 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7787949c76-lxf6c"] Jun 03 01:42:41.959806 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:41.959776 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7787949c76-lxf6c" podUID="4a9af881-d17b-45be-a889-1f887e0ca248" Jun 03 01:42:41.963133 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.963107 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh"] Jun 03 01:42:41.964119 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:41.964099 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv"] Jun 03 01:42:42.099561 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.099499 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/df01c07c-bdfa-4a36-948f-cffe32a31c79-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-p9ffv\" (UID: \"df01c07c-bdfa-4a36-948f-cffe32a31c79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:42.099708 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.099581 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg648\" (UniqueName: \"kubernetes.io/projected/4476b4b9-86c6-4c55-95ef-ae4524693eaf-kube-api-access-pg648\") pod \"network-check-source-56d84b9f44-x45hh\" (UID: \"4476b4b9-86c6-4c55-95ef-ae4524693eaf\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" Jun 03 01:42:42.200573 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.200515 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pg648\" (UniqueName: \"kubernetes.io/projected/4476b4b9-86c6-4c55-95ef-ae4524693eaf-kube-api-access-pg648\") pod \"network-check-source-56d84b9f44-x45hh\" (UID: \"4476b4b9-86c6-4c55-95ef-ae4524693eaf\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" Jun 03 01:42:42.200700 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.200656 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/df01c07c-bdfa-4a36-948f-cffe32a31c79-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-p9ffv\" (UID: \"df01c07c-bdfa-4a36-948f-cffe32a31c79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:42.203251 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.203229 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/df01c07c-bdfa-4a36-948f-cffe32a31c79-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-p9ffv\" (UID: \"df01c07c-bdfa-4a36-948f-cffe32a31c79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:42.215183 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.215154 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg648\" (UniqueName: \"kubernetes.io/projected/4476b4b9-86c6-4c55-95ef-ae4524693eaf-kube-api-access-pg648\") pod \"network-check-source-56d84b9f44-x45hh\" (UID: \"4476b4b9-86c6-4c55-95ef-ae4524693eaf\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" Jun 03 01:42:42.264133 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.264097 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:42.269893 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.269864 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" Jun 03 01:42:42.398847 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.398823 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh"] Jun 03 01:42:42.401254 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:42.401222 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4476b4b9_86c6_4c55_95ef_ae4524693eaf.slice/crio-4f790bea03c699c6307913593ba195572aaca0686aa9f22dd34d79d2a997aa67 WatchSource:0}: Error finding container 4f790bea03c699c6307913593ba195572aaca0686aa9f22dd34d79d2a997aa67: Status 404 returned error can't find the container with id 4f790bea03c699c6307913593ba195572aaca0686aa9f22dd34d79d2a997aa67 Jun 03 01:42:42.415945 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.415924 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv"] Jun 03 01:42:42.418747 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:42.418718 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf01c07c_bdfa_4a36_948f_cffe32a31c79.slice/crio-f5a98c3e00ba05004ce7dcf633b3a7d7f59ecff56cec73c96372fd8293884ecf WatchSource:0}: Error finding container f5a98c3e00ba05004ce7dcf633b3a7d7f59ecff56cec73c96372fd8293884ecf: Status 404 returned error can't find the container with id f5a98c3e00ba05004ce7dcf633b3a7d7f59ecff56cec73c96372fd8293884ecf Jun 03 01:42:42.421414 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.421389 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" event={"ID":"4476b4b9-86c6-4c55-95ef-ae4524693eaf","Type":"ContainerStarted","Data":"4f790bea03c699c6307913593ba195572aaca0686aa9f22dd34d79d2a997aa67"} Jun 03 01:42:42.421474 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.421427 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:42:42.434045 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.434020 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:42:42.503599 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503561 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-installation-pull-secrets\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.503785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503613 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-bound-sa-token\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.503785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503676 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-trusted-ca\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.503785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503719 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-image-registry-private-configuration\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.503785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503743 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk7l9\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-kube-api-access-zk7l9\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.503785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503769 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-registry-certificates\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.504035 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.503804 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a9af881-d17b-45be-a889-1f887e0ca248-ca-trust-extracted\") pod \"4a9af881-d17b-45be-a889-1f887e0ca248\" (UID: \"4a9af881-d17b-45be-a889-1f887e0ca248\") " Jun 03 01:42:42.504189 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.504134 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:42:42.504262 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.504207 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9af881-d17b-45be-a889-1f887e0ca248-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 01:42:42.504262 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.504219 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:42:42.506037 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.505999 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:42:42.506241 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.506224 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:42:42.506516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.506490 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:42:42.506634 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.506578 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-kube-api-access-zk7l9" (OuterVolumeSpecName: "kube-api-access-zk7l9") pod "4a9af881-d17b-45be-a889-1f887e0ca248" (UID: "4a9af881-d17b-45be-a889-1f887e0ca248"). InnerVolumeSpecName "kube-api-access-zk7l9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:42:42.605030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.604994 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-trusted-ca\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:42.605030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.605025 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-image-registry-private-configuration\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:42.605030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.605036 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk7l9\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-kube-api-access-zk7l9\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:42.605290 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.605046 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a9af881-d17b-45be-a889-1f887e0ca248-registry-certificates\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:42.605290 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.605056 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a9af881-d17b-45be-a889-1f887e0ca248-ca-trust-extracted\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:42.605290 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.605065 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a9af881-d17b-45be-a889-1f887e0ca248-installation-pull-secrets\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:42.605290 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:42.605073 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-bound-sa-token\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:43.286350 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.286304 2553 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:43.286350 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.286352 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:43.286933 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.286811 2553 scope.go:117] "RemoveContainer" containerID="d18160cfba22431090f928602f9ab1719afe4c9518e08a8de0137b9b7b5dc271" Jun 03 01:42:43.287016 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:43.286997 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-xxmhj_openshift-console-operator(b16cd5f5-9216-4743-a5ab-97375ec370c1)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" podUID="b16cd5f5-9216-4743-a5ab-97375ec370c1" Jun 03 01:42:43.428313 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.428273 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" event={"ID":"4476b4b9-86c6-4c55-95ef-ae4524693eaf","Type":"ContainerStarted","Data":"4cbe8d9439a7ae9ab0d9838d98fcb582abcd5f47fb6f4b8eac9beaf656793e13"} Jun 03 01:42:43.429479 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.429446 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" event={"ID":"df01c07c-bdfa-4a36-948f-cffe32a31c79","Type":"ContainerStarted","Data":"f5a98c3e00ba05004ce7dcf633b3a7d7f59ecff56cec73c96372fd8293884ecf"} Jun 03 01:42:43.429617 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.429502 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7787949c76-lxf6c" Jun 03 01:42:43.445235 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.445177 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-x45hh" podStartSLOduration=2.445156261 podStartE2EDuration="2.445156261s" podCreationTimestamp="2026-06-03 01:42:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:42:43.443791302 +0000 UTC m=+134.098434583" watchObservedRunningTime="2026-06-03 01:42:43.445156261 +0000 UTC m=+134.099799541" Jun 03 01:42:43.475755 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.475669 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7787949c76-lxf6c"] Jun 03 01:42:43.478865 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.478837 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7787949c76-lxf6c"] Jun 03 01:42:43.512418 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.512374 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a9af881-d17b-45be-a889-1f887e0ca248-registry-tls\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:42:43.938649 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:43.938617 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a9af881-d17b-45be-a889-1f887e0ca248" path="/var/lib/kubelet/pods/4a9af881-d17b-45be-a889-1f887e0ca248/volumes" Jun 03 01:42:44.434275 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:44.434232 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" event={"ID":"df01c07c-bdfa-4a36-948f-cffe32a31c79","Type":"ContainerStarted","Data":"ebfcf17a5b977dc61f7dd3235cade29566d3efe7431e666e6b303b58786d2536"} Jun 03 01:42:44.434691 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:44.434396 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:44.439057 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:44.439034 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" Jun 03 01:42:44.449789 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:44.449744 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-p9ffv" podStartSLOduration=2.282425152 podStartE2EDuration="3.449728483s" podCreationTimestamp="2026-06-03 01:42:41 +0000 UTC" firstStartedPulling="2026-06-03 01:42:42.420570405 +0000 UTC m=+133.075213663" lastFinishedPulling="2026-06-03 01:42:43.587873722 +0000 UTC m=+134.242516994" observedRunningTime="2026-06-03 01:42:44.44869525 +0000 UTC m=+135.103338530" watchObservedRunningTime="2026-06-03 01:42:44.449728483 +0000 UTC m=+135.104371764" Jun 03 01:42:45.199455 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.199419 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm"] Jun 03 01:42:45.205604 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.205586 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.209375 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.209348 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 01:42:45.209375 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.209365 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 03 01:42:45.209375 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.209349 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2jcdh\"" Jun 03 01:42:45.209667 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.209414 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 01:42:45.209667 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.209350 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 03 01:42:45.209667 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.209461 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 01:42:45.212695 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.212676 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm"] Jun 03 01:42:45.222859 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.222827 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwvjl\" (UniqueName: \"kubernetes.io/projected/d86b384b-923c-468d-9f67-6908e2065022-kube-api-access-cwvjl\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.223011 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.222876 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d86b384b-923c-468d-9f67-6908e2065022-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.223011 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.222897 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d86b384b-923c-468d-9f67-6908e2065022-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.223011 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.222916 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d86b384b-923c-468d-9f67-6908e2065022-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.323679 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.323644 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cwvjl\" (UniqueName: \"kubernetes.io/projected/d86b384b-923c-468d-9f67-6908e2065022-kube-api-access-cwvjl\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.323878 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.323703 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d86b384b-923c-468d-9f67-6908e2065022-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.323878 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.323727 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d86b384b-923c-468d-9f67-6908e2065022-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.323878 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.323748 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d86b384b-923c-468d-9f67-6908e2065022-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.324449 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.324424 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d86b384b-923c-468d-9f67-6908e2065022-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.326261 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.326244 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d86b384b-923c-468d-9f67-6908e2065022-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.326320 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.326258 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d86b384b-923c-468d-9f67-6908e2065022-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.332920 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.332892 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwvjl\" (UniqueName: \"kubernetes.io/projected/d86b384b-923c-468d-9f67-6908e2065022-kube-api-access-cwvjl\") pod \"prometheus-operator-658bcd8bdf-dxdzm\" (UID: \"d86b384b-923c-468d-9f67-6908e2065022\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.514923 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.514826 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" Jun 03 01:42:45.637875 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:45.637797 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm"] Jun 03 01:42:45.639886 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:45.639865 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86b384b_923c_468d_9f67_6908e2065022.slice/crio-4c4822d3e8bd50553daa20bb60f82920657f728463a22d3591ed0378b059cfaa WatchSource:0}: Error finding container 4c4822d3e8bd50553daa20bb60f82920657f728463a22d3591ed0378b059cfaa: Status 404 returned error can't find the container with id 4c4822d3e8bd50553daa20bb60f82920657f728463a22d3591ed0378b059cfaa Jun 03 01:42:46.440423 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:46.440380 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" event={"ID":"d86b384b-923c-468d-9f67-6908e2065022","Type":"ContainerStarted","Data":"4c4822d3e8bd50553daa20bb60f82920657f728463a22d3591ed0378b059cfaa"} Jun 03 01:42:47.446596 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:47.446558 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" event={"ID":"d86b384b-923c-468d-9f67-6908e2065022","Type":"ContainerStarted","Data":"3afe41aae2a671606e3bc3f9175ad16a098814747252e6bb8b5c04a363e498fb"} Jun 03 01:42:47.446596 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:47.446599 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" event={"ID":"d86b384b-923c-468d-9f67-6908e2065022","Type":"ContainerStarted","Data":"132d67a561a34494df960f840082b0806f783963f76e1fb6e53d71bb549db043"} Jun 03 01:42:47.464105 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:47.464057 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dxdzm" podStartSLOduration=1.144258274 podStartE2EDuration="2.464042878s" podCreationTimestamp="2026-06-03 01:42:45 +0000 UTC" firstStartedPulling="2026-06-03 01:42:45.641711002 +0000 UTC m=+136.296354260" lastFinishedPulling="2026-06-03 01:42:46.961495592 +0000 UTC m=+137.616138864" observedRunningTime="2026-06-03 01:42:47.463593638 +0000 UTC m=+138.118236920" watchObservedRunningTime="2026-06-03 01:42:47.464042878 +0000 UTC m=+138.118686157" Jun 03 01:42:49.538022 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.537982 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-f75sv"] Jun 03 01:42:49.545804 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.545779 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.548494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.548289 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 03 01:42:49.548494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.548404 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kz49k\"" Jun 03 01:42:49.548719 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.548686 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 03 01:42:49.549146 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.549119 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-f75sv"] Jun 03 01:42:49.551764 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.551740 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp"] Jun 03 01:42:49.552852 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.552830 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.552989 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.552892 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.552989 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.552922 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.552989 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.552970 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrpd\" (UniqueName: \"kubernetes.io/projected/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-kube-api-access-lsrpd\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.554990 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.554972 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.557620 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.557602 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 03 01:42:49.557718 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.557679 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 03 01:42:49.557718 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.557679 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-886cp\"" Jun 03 01:42:49.557980 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.557966 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 03 01:42:49.565325 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.565297 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp"] Jun 03 01:42:49.586185 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.586159 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-86spt"] Jun 03 01:42:49.589512 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.589497 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.592008 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.591986 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 01:42:49.592354 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.592331 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 01:42:49.592448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.592331 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 01:42:49.592448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.592362 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f77s6\"" Jun 03 01:42:49.653825 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.653794 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-metrics-client-ca\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654025 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.653833 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22q44\" (UniqueName: \"kubernetes.io/projected/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-api-access-22q44\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.654025 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.653861 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrpd\" (UniqueName: \"kubernetes.io/projected/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-kube-api-access-lsrpd\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.654025 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.653914 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.654025 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.653956 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654177 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654026 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654177 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654082 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-sys\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654177 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654107 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.654177 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654128 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5fd2a5d-fb65-4287-9a46-6444766e670b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.654177 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654158 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654187 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654225 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-wtmp\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654249 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-root\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-textfile\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654302 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654327 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.654366 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654360 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-tls\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654731 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654392 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5fd2a5d-fb65-4287-9a46-6444766e670b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.654731 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654417 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5v85\" (UniqueName: \"kubernetes.io/projected/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-kube-api-access-c5v85\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.654929 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.654907 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.656890 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.656868 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.656992 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.656945 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.661335 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.661312 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrpd\" (UniqueName: \"kubernetes.io/projected/0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e-kube-api-access-lsrpd\") pod \"openshift-state-metrics-85658796db-f75sv\" (UID: \"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.755662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755622 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-textfile\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.755662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755668 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-tls\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755698 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5fd2a5d-fb65-4287-9a46-6444766e670b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755719 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c5v85\" (UniqueName: \"kubernetes.io/projected/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-kube-api-access-c5v85\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755746 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-metrics-client-ca\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755773 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-22q44\" (UniqueName: \"kubernetes.io/projected/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-api-access-22q44\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755800 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755825 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.755913 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755896 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755926 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-sys\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.755966 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756000 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5fd2a5d-fb65-4287-9a46-6444766e670b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756020 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-textfile\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756036 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756094 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-wtmp\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756123 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-root\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756198 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-root\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756277 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-wtmp\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756329 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-sys\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756370 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-metrics-client-ca\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:49.756406 2553 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:42:49.756469 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-tls podName:b5fd2a5d-fb65-4287-9a46-6444766e670b nodeName:}" failed. No retries permitted until 2026-06-03 01:42:50.256451762 +0000 UTC m=+140.911095024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-qdqbp" (UID: "b5fd2a5d-fb65-4287-9a46-6444766e670b") : secret "kube-state-metrics-tls" not found Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756478 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5fd2a5d-fb65-4287-9a46-6444766e670b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.756853 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756842 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.757268 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.756988 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5fd2a5d-fb65-4287-9a46-6444766e670b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.757268 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.757062 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.758436 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.758408 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-tls\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.758573 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.758556 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.758647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.758631 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.764326 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.764305 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5v85\" (UniqueName: \"kubernetes.io/projected/9b1aac16-1196-4e6c-a811-b5b02e2c08e1-kube-api-access-c5v85\") pod \"node-exporter-86spt\" (UID: \"9b1aac16-1196-4e6c-a811-b5b02e2c08e1\") " pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.764742 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.764720 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22q44\" (UniqueName: \"kubernetes.io/projected/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-api-access-22q44\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:49.856832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.856798 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" Jun 03 01:42:49.898878 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.898844 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-86spt" Jun 03 01:42:49.908639 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:49.908602 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1aac16_1196_4e6c_a811_b5b02e2c08e1.slice/crio-d8c956c93d27fbed7e745302fe07f0831d2c91f34d41afc06ed13b28195710bb WatchSource:0}: Error finding container d8c956c93d27fbed7e745302fe07f0831d2c91f34d41afc06ed13b28195710bb: Status 404 returned error can't find the container with id d8c956c93d27fbed7e745302fe07f0831d2c91f34d41afc06ed13b28195710bb Jun 03 01:42:49.990384 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:49.990347 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-f75sv"] Jun 03 01:42:49.993054 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:49.993027 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe4f4ab_c16a_4b1c_bef1_4effcd5c4f6e.slice/crio-55aa467f050f6f82bc52b7f10e215801fb934339957a5cc7dc01114745fa707f WatchSource:0}: Error finding container 55aa467f050f6f82bc52b7f10e215801fb934339957a5cc7dc01114745fa707f: Status 404 returned error can't find the container with id 55aa467f050f6f82bc52b7f10e215801fb934339957a5cc7dc01114745fa707f Jun 03 01:42:50.260998 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.260969 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:50.263951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.263702 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5fd2a5d-fb65-4287-9a46-6444766e670b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-qdqbp\" (UID: \"b5fd2a5d-fb65-4287-9a46-6444766e670b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:50.455776 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.455667 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86spt" event={"ID":"9b1aac16-1196-4e6c-a811-b5b02e2c08e1","Type":"ContainerStarted","Data":"d8c956c93d27fbed7e745302fe07f0831d2c91f34d41afc06ed13b28195710bb"} Jun 03 01:42:50.457680 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.457653 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" event={"ID":"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e","Type":"ContainerStarted","Data":"fa39250d78d46d62b76d2974aa33482b39d359c43ac04a68c9148d27a86c695e"} Jun 03 01:42:50.457783 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.457687 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" event={"ID":"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e","Type":"ContainerStarted","Data":"f184400b26d8fb6b9177b436fba65cf2fa8a8796c081be557ed08dc65d5b0128"} Jun 03 01:42:50.457783 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.457703 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" event={"ID":"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e","Type":"ContainerStarted","Data":"55aa467f050f6f82bc52b7f10e215801fb934339957a5cc7dc01114745fa707f"} Jun 03 01:42:50.465049 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.465017 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" Jun 03 01:42:50.628862 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:50.628823 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp"] Jun 03 01:42:50.631965 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:50.631929 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5fd2a5d_fb65_4287_9a46_6444766e670b.slice/crio-94ebdd92f6fed9c232dc51bd084bcac0de669708e4a673b5c07543f82c443a3d WatchSource:0}: Error finding container 94ebdd92f6fed9c232dc51bd084bcac0de669708e4a673b5c07543f82c443a3d: Status 404 returned error can't find the container with id 94ebdd92f6fed9c232dc51bd084bcac0de669708e4a673b5c07543f82c443a3d Jun 03 01:42:51.462629 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.462586 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" event={"ID":"b5fd2a5d-fb65-4287-9a46-6444766e670b","Type":"ContainerStarted","Data":"94ebdd92f6fed9c232dc51bd084bcac0de669708e4a673b5c07543f82c443a3d"} Jun 03 01:42:51.516789 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.516685 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-85b6f6778d-9wpr8"] Jun 03 01:42:51.536179 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.536142 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85b6f6778d-9wpr8"] Jun 03 01:42:51.536356 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.536331 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.539459 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539252 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 03 01:42:51.539459 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539252 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 03 01:42:51.539459 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539314 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 03 01:42:51.539459 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539367 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 03 01:42:51.539809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539672 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-784rcgvfsttd8\"" Jun 03 01:42:51.539809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539689 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 03 01:42:51.539910 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.539829 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hnhfb\"" Jun 03 01:42:51.573386 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573342 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573620 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573484 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-tls\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573620 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573555 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-grpc-tls\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573757 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmplr\" (UniqueName: \"kubernetes.io/projected/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-kube-api-access-gmplr\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573757 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573657 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573757 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573734 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573875 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573794 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.573875 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.573829 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-metrics-client-ca\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674349 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674315 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-tls\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674368 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-grpc-tls\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674607 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gmplr\" (UniqueName: \"kubernetes.io/projected/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-kube-api-access-gmplr\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674655 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674702 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674769 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674798 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-metrics-client-ca\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.674872 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.674840 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.676166 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.676139 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-metrics-client-ca\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.677960 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.677923 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-tls\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.678260 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.678236 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-grpc-tls\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.678370 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.678304 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.678472 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.678439 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.678560 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.678495 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.680204 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.679778 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.683619 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.683099 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmplr\" (UniqueName: \"kubernetes.io/projected/79d0642a-f890-42dc-85e9-ca43ec3bbfcc-kube-api-access-gmplr\") pod \"thanos-querier-85b6f6778d-9wpr8\" (UID: \"79d0642a-f890-42dc-85e9-ca43ec3bbfcc\") " pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.850297 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.850152 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:51.997487 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:51.997462 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85b6f6778d-9wpr8"] Jun 03 01:42:52.295795 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:52.295724 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d0642a_f890_42dc_85e9_ca43ec3bbfcc.slice/crio-06545472928c2b846f0b7699c72ee39f4ad3e4b4b19686cd13ef345343780e62 WatchSource:0}: Error finding container 06545472928c2b846f0b7699c72ee39f4ad3e4b4b19686cd13ef345343780e62: Status 404 returned error can't find the container with id 06545472928c2b846f0b7699c72ee39f4ad3e4b4b19686cd13ef345343780e62 Jun 03 01:42:52.469565 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:52.468515 2553 generic.go:358] "Generic (PLEG): container finished" podID="9b1aac16-1196-4e6c-a811-b5b02e2c08e1" containerID="7445b3ff655afd61c1c251cb6e7b2769c640ed893a683c0c9ffee30657648b9a" exitCode=0 Jun 03 01:42:52.469565 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:52.468627 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86spt" event={"ID":"9b1aac16-1196-4e6c-a811-b5b02e2c08e1","Type":"ContainerDied","Data":"7445b3ff655afd61c1c251cb6e7b2769c640ed893a683c0c9ffee30657648b9a"} Jun 03 01:42:52.476695 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:52.474389 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" event={"ID":"0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e","Type":"ContainerStarted","Data":"d9ad923142c4981df612d1987cf5e0b15000910b5a4ec6a1975653871897f536"} Jun 03 01:42:52.480437 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:52.480389 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" event={"ID":"b5fd2a5d-fb65-4287-9a46-6444766e670b","Type":"ContainerStarted","Data":"5c5c0f97d3d237f5ceb1eef9c1fe6271763137f93ec8fba44c284318c4a259e4"} Jun 03 01:42:52.481857 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:52.481823 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"06545472928c2b846f0b7699c72ee39f4ad3e4b4b19686cd13ef345343780e62"} Jun 03 01:42:52.508766 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:52.508447 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-f75sv" podStartSLOduration=1.983553328 podStartE2EDuration="3.508426767s" podCreationTimestamp="2026-06-03 01:42:49 +0000 UTC" firstStartedPulling="2026-06-03 01:42:50.249158657 +0000 UTC m=+140.903801918" lastFinishedPulling="2026-06-03 01:42:51.774032085 +0000 UTC m=+142.428675357" observedRunningTime="2026-06-03 01:42:52.507561032 +0000 UTC m=+143.162204304" watchObservedRunningTime="2026-06-03 01:42:52.508426767 +0000 UTC m=+143.163070120" Jun 03 01:42:53.487385 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.487343 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86spt" event={"ID":"9b1aac16-1196-4e6c-a811-b5b02e2c08e1","Type":"ContainerStarted","Data":"1edbe6c5a1efb76ca056a9dfe57fa3a4fb60a81852916a0496e3190adb859631"} Jun 03 01:42:53.487385 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.487390 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86spt" event={"ID":"9b1aac16-1196-4e6c-a811-b5b02e2c08e1","Type":"ContainerStarted","Data":"f9ee52500fb1515d10ea450cae15047b071835c8b6ec7493a6c21d2640da3211"} Jun 03 01:42:53.489551 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.489501 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" event={"ID":"b5fd2a5d-fb65-4287-9a46-6444766e670b","Type":"ContainerStarted","Data":"855e377c2a9b3422c5f56a66216b193a6468274a60526bd278ca4bc3381c9fbf"} Jun 03 01:42:53.489694 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.489562 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" event={"ID":"b5fd2a5d-fb65-4287-9a46-6444766e670b","Type":"ContainerStarted","Data":"9a7b86fa0e454e84217d81bd9b874037e33a4a62a4dd00bd7602761eccc92195"} Jun 03 01:42:53.509547 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.509467 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-86spt" podStartSLOduration=2.7795362519999998 podStartE2EDuration="4.509449952s" podCreationTimestamp="2026-06-03 01:42:49 +0000 UTC" firstStartedPulling="2026-06-03 01:42:49.910742177 +0000 UTC m=+140.565385436" lastFinishedPulling="2026-06-03 01:42:51.640655874 +0000 UTC m=+142.295299136" observedRunningTime="2026-06-03 01:42:53.50704757 +0000 UTC m=+144.161690849" watchObservedRunningTime="2026-06-03 01:42:53.509449952 +0000 UTC m=+144.164093257" Jun 03 01:42:53.526453 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.526399 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-qdqbp" podStartSLOduration=2.827156448 podStartE2EDuration="4.526385359s" podCreationTimestamp="2026-06-03 01:42:49 +0000 UTC" firstStartedPulling="2026-06-03 01:42:50.633953668 +0000 UTC m=+141.288596925" lastFinishedPulling="2026-06-03 01:42:52.333182563 +0000 UTC m=+142.987825836" observedRunningTime="2026-06-03 01:42:53.525682138 +0000 UTC m=+144.180325419" watchObservedRunningTime="2026-06-03 01:42:53.526385359 +0000 UTC m=+144.181028636" Jun 03 01:42:53.944639 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.944593 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5fb755fc7-zjx28"] Jun 03 01:42:53.948000 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.947971 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.952881 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.952836 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 03 01:42:53.953554 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.953495 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 03 01:42:53.953675 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.953641 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cjovkc4bghist\"" Jun 03 01:42:53.954027 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.953937 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-g4lhk\"" Jun 03 01:42:53.954321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.954306 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 03 01:42:53.954982 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.954306 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 03 01:42:53.958714 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.958688 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fb755fc7-zjx28"] Jun 03 01:42:53.996734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996689 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6lvz\" (UniqueName: \"kubernetes.io/projected/be4bb435-e6b7-4b28-86f9-4d7d059b452f-kube-api-access-r6lvz\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.996916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996745 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-secret-metrics-server-tls\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.996916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996799 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/be4bb435-e6b7-4b28-86f9-4d7d059b452f-metrics-server-audit-profiles\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.996916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996827 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be4bb435-e6b7-4b28-86f9-4d7d059b452f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.996916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996852 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/be4bb435-e6b7-4b28-86f9-4d7d059b452f-audit-log\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.996916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996891 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-client-ca-bundle\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:53.997100 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:53.996924 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-secret-metrics-server-client-certs\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.097994 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.097933 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-secret-metrics-server-tls\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098028 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/be4bb435-e6b7-4b28-86f9-4d7d059b452f-metrics-server-audit-profiles\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098061 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be4bb435-e6b7-4b28-86f9-4d7d059b452f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098088 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/be4bb435-e6b7-4b28-86f9-4d7d059b452f-audit-log\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098110 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-client-ca-bundle\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098137 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-secret-metrics-server-client-certs\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098449 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098303 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r6lvz\" (UniqueName: \"kubernetes.io/projected/be4bb435-e6b7-4b28-86f9-4d7d059b452f-kube-api-access-r6lvz\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.098654 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.098623 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/be4bb435-e6b7-4b28-86f9-4d7d059b452f-audit-log\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.099404 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.099374 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be4bb435-e6b7-4b28-86f9-4d7d059b452f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.099571 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.099525 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/be4bb435-e6b7-4b28-86f9-4d7d059b452f-metrics-server-audit-profiles\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.101249 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.101228 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-client-ca-bundle\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.101348 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.101248 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-secret-metrics-server-client-certs\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.101419 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.101400 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/be4bb435-e6b7-4b28-86f9-4d7d059b452f-secret-metrics-server-tls\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.106726 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.106683 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6lvz\" (UniqueName: \"kubernetes.io/projected/be4bb435-e6b7-4b28-86f9-4d7d059b452f-kube-api-access-r6lvz\") pod \"metrics-server-5fb755fc7-zjx28\" (UID: \"be4bb435-e6b7-4b28-86f9-4d7d059b452f\") " pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.265183 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.265073 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:42:54.319444 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.318730 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq"] Jun 03 01:42:54.322648 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.322617 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:54.325328 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.325301 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 03 01:42:54.325328 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.325323 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-shc6m\"" Jun 03 01:42:54.330171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.330136 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq"] Jun 03 01:42:54.401392 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.401353 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/84da1117-279d-42b4-92d2-831d29e246c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-2cngq\" (UID: \"84da1117-279d-42b4-92d2-831d29e246c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:54.430576 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.430491 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fb755fc7-zjx28"] Jun 03 01:42:54.502187 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.502150 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/84da1117-279d-42b4-92d2-831d29e246c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-2cngq\" (UID: \"84da1117-279d-42b4-92d2-831d29e246c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:54.505299 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.505268 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/84da1117-279d-42b4-92d2-831d29e246c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-2cngq\" (UID: \"84da1117-279d-42b4-92d2-831d29e246c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:54.639630 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.639596 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:54.864475 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:54.864435 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4bb435_e6b7_4b28_86f9_4d7d059b452f.slice/crio-6dee301a20effb7df07313d1055fb922f2de6ad3c117eb1f32fe707c856f7b98 WatchSource:0}: Error finding container 6dee301a20effb7df07313d1055fb922f2de6ad3c117eb1f32fe707c856f7b98: Status 404 returned error can't find the container with id 6dee301a20effb7df07313d1055fb922f2de6ad3c117eb1f32fe707c856f7b98 Jun 03 01:42:54.998731 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:54.998697 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq"] Jun 03 01:42:55.001752 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:55.001719 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84da1117_279d_42b4_92d2_831d29e246c8.slice/crio-1f21e331c589618defc672413977ddad2c77b0fa2c7b948988215971e15da3f0 WatchSource:0}: Error finding container 1f21e331c589618defc672413977ddad2c77b0fa2c7b948988215971e15da3f0: Status 404 returned error can't find the container with id 1f21e331c589618defc672413977ddad2c77b0fa2c7b948988215971e15da3f0 Jun 03 01:42:55.501481 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.501439 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"2a9afcd02f1bdc0bae89a47f4f0af794926f9d033f2409c9797c835b4a4445b5"} Jun 03 01:42:55.501677 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.501489 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"7355fbf00b8f81df23c1e4f57648414ed3e38c6cc42a903eea14e90a9fecca54"} Jun 03 01:42:55.501677 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.501502 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"141db2d249adae2cfe13ca7a8c7c151f6d2dcc4941853784add10528d96a1357"} Jun 03 01:42:55.503290 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.503245 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" event={"ID":"be4bb435-e6b7-4b28-86f9-4d7d059b452f","Type":"ContainerStarted","Data":"6dee301a20effb7df07313d1055fb922f2de6ad3c117eb1f32fe707c856f7b98"} Jun 03 01:42:55.504652 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.504611 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" event={"ID":"84da1117-279d-42b4-92d2-831d29e246c8","Type":"ContainerStarted","Data":"1f21e331c589618defc672413977ddad2c77b0fa2c7b948988215971e15da3f0"} Jun 03 01:42:55.782751 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.782636 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:42:55.807499 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.802362 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.814550 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.812882 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 03 01:42:55.814550 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.813178 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 03 01:42:55.814550 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.813398 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 03 01:42:55.818576 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.815310 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 03 01:42:55.818576 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.815609 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 03 01:42:55.818576 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.815877 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 03 01:42:55.818576 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.816100 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 03 01:42:55.821076 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.821051 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:42:55.821689 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.821668 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 03 01:42:55.822161 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.822128 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 03 01:42:55.822588 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.822573 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bstn5\"" Jun 03 01:42:55.823042 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.823026 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4ptuc8tl6tr3p\"" Jun 03 01:42:55.823501 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.823485 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 03 01:42:55.826228 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.826202 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 03 01:42:55.829312 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.829290 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 03 01:42:55.922267 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922158 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922457 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922406 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-web-config\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922509 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922469 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922674 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922650 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-config\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922735 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922723 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922815 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922800 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922875 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922830 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.922979 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.922953 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923048 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923033 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7m6d\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-kube-api-access-b7m6d\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923172 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923111 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923258 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923240 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923332 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923316 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923377 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923351 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923506 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923425 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923506 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923490 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923644 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923579 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923693 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923656 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-config-out\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:55.923739 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:55.923713 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024685 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-web-config\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024738 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024785 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-config\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024811 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024849 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024874 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024907 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024936 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b7m6d\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-kube-api-access-b7m6d\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.024971 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025014 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025055 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025089 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025114 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025153 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025190 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025221 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-config-out\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.025448 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025248 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.026392 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025280 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.026392 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.025727 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.027553 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.027220 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.028170 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.028140 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.031411 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.030825 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-config\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.032181 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.032153 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-config-out\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.035826 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.035463 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.037162 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.036089 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.037162 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.036261 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.037162 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.036465 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.037162 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.036876 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.037162 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.037129 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.037785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.037737 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.041937 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.041868 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.041937 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.041931 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.042253 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.042166 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.042603 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.042569 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.048725 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.048691 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7m6d\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-kube-api-access-b7m6d\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.048852 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.048789 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-web-config\") pod \"prometheus-k8s-0\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.153360 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.153324 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:42:56.510301 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.510252 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"e92ce38c5b4bb30c02db3e254fb7cbb8989fa9a45c28e9b88716fbdb5ceaf248"} Jun 03 01:42:56.934974 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:56.934951 2553 scope.go:117] "RemoveContainer" containerID="d18160cfba22431090f928602f9ab1719afe4c9518e08a8de0137b9b7b5dc271" Jun 03 01:42:57.095569 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.095516 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:42:57.097250 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:42:57.097208 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef28bab_85f7_4094_a9b7_f4458222964c.slice/crio-62e8162e2f8ea9090ebccf2b2fee2b239ee6d2f08f0f593bb2b263e6c6863a15 WatchSource:0}: Error finding container 62e8162e2f8ea9090ebccf2b2fee2b239ee6d2f08f0f593bb2b263e6c6863a15: Status 404 returned error can't find the container with id 62e8162e2f8ea9090ebccf2b2fee2b239ee6d2f08f0f593bb2b263e6c6863a15 Jun 03 01:42:57.517155 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.517114 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"7aef0b42b48eca7946e26b205f764f7d10280259aa0ed26724148809704a7745"} Jun 03 01:42:57.517155 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.517162 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" event={"ID":"79d0642a-f890-42dc-85e9-ca43ec3bbfcc","Type":"ContainerStarted","Data":"feaa5704d8fb7ee09f60f3bc244680d4c4fcf1113a2aa1fc30c1a34c6f03d84c"} Jun 03 01:42:57.517695 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.517390 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:42:57.519997 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.519977 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:42:57.520092 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.520063 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" event={"ID":"b16cd5f5-9216-4743-a5ab-97375ec370c1","Type":"ContainerStarted","Data":"74e63681f70907f9aeabe1247976c8c76fca99a145484c1e1d56a5483d0d2bc9"} Jun 03 01:42:57.520367 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.520342 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:57.521810 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.521781 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" event={"ID":"be4bb435-e6b7-4b28-86f9-4d7d059b452f","Type":"ContainerStarted","Data":"486f491deb27c95822a8f0f0353c49165d8077be4e5e423e13bd0ac462eb7828"} Jun 03 01:42:57.523844 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.523815 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" event={"ID":"84da1117-279d-42b4-92d2-831d29e246c8","Type":"ContainerStarted","Data":"252baace0e5d5f327d0990dd37f1683e03525c3a5ef25564bb451e08b50c10ca"} Jun 03 01:42:57.524148 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.523969 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:57.525208 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.525190 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"62e8162e2f8ea9090ebccf2b2fee2b239ee6d2f08f0f593bb2b263e6c6863a15"} Jun 03 01:42:57.530446 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.530422 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" Jun 03 01:42:57.542113 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.542057 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" podStartSLOduration=2.7533708150000002 podStartE2EDuration="6.542038218s" podCreationTimestamp="2026-06-03 01:42:51 +0000 UTC" firstStartedPulling="2026-06-03 01:42:52.297602713 +0000 UTC m=+142.952245970" lastFinishedPulling="2026-06-03 01:42:56.086270115 +0000 UTC m=+146.740913373" observedRunningTime="2026-06-03 01:42:57.540052986 +0000 UTC m=+148.194696267" watchObservedRunningTime="2026-06-03 01:42:57.542038218 +0000 UTC m=+148.196681499" Jun 03 01:42:57.554214 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.554102 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-2cngq" podStartSLOduration=1.622863414 podStartE2EDuration="3.554086287s" podCreationTimestamp="2026-06-03 01:42:54 +0000 UTC" firstStartedPulling="2026-06-03 01:42:55.00412147 +0000 UTC m=+145.658764732" lastFinishedPulling="2026-06-03 01:42:56.935344346 +0000 UTC m=+147.589987605" observedRunningTime="2026-06-03 01:42:57.553337177 +0000 UTC m=+148.207980457" watchObservedRunningTime="2026-06-03 01:42:57.554086287 +0000 UTC m=+148.208729615" Jun 03 01:42:57.591216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.591152 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" podStartSLOduration=43.088984469 podStartE2EDuration="45.591130794s" podCreationTimestamp="2026-06-03 01:42:12 +0000 UTC" firstStartedPulling="2026-06-03 01:42:13.440434556 +0000 UTC m=+104.095077815" lastFinishedPulling="2026-06-03 01:42:15.942580882 +0000 UTC m=+106.597224140" observedRunningTime="2026-06-03 01:42:57.572636821 +0000 UTC m=+148.227280102" watchObservedRunningTime="2026-06-03 01:42:57.591130794 +0000 UTC m=+148.245774075" Jun 03 01:42:57.591383 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.591271 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" podStartSLOduration=2.526281719 podStartE2EDuration="4.591264186s" podCreationTimestamp="2026-06-03 01:42:53 +0000 UTC" firstStartedPulling="2026-06-03 01:42:54.868896391 +0000 UTC m=+145.523539649" lastFinishedPulling="2026-06-03 01:42:56.933878843 +0000 UTC m=+147.588522116" observedRunningTime="2026-06-03 01:42:57.589265306 +0000 UTC m=+148.243908655" watchObservedRunningTime="2026-06-03 01:42:57.591264186 +0000 UTC m=+148.245907467" Jun 03 01:42:57.800422 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:57.800346 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-xxmhj" Jun 03 01:42:58.529498 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:58.529409 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" exitCode=0 Jun 03 01:42:58.529906 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:42:58.529491 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} Jun 03 01:43:02.545714 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.545677 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} Jun 03 01:43:02.545714 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.545715 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} Jun 03 01:43:02.546145 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.545725 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} Jun 03 01:43:02.546145 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.545734 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} Jun 03 01:43:02.546145 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.545745 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} Jun 03 01:43:02.546145 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.545754 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerStarted","Data":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} Jun 03 01:43:02.575111 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:02.575049 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.172449257 podStartE2EDuration="7.575030194s" podCreationTimestamp="2026-06-03 01:42:55 +0000 UTC" firstStartedPulling="2026-06-03 01:42:57.099724934 +0000 UTC m=+147.754368197" lastFinishedPulling="2026-06-03 01:43:01.502305873 +0000 UTC m=+152.156949134" observedRunningTime="2026-06-03 01:43:02.572726674 +0000 UTC m=+153.227369955" watchObservedRunningTime="2026-06-03 01:43:02.575030194 +0000 UTC m=+153.229673475" Jun 03 01:43:03.536231 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:03.536198 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-85b6f6778d-9wpr8" Jun 03 01:43:06.154446 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:06.154414 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:43:06.693855 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:43:06.693812 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bwd6s" podUID="74c31246-3616-41a4-b95a-0a5242f1fa28" Jun 03 01:43:06.702030 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:43:06.701996 2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-x4r47" podUID="cc11711b-16fb-4148-bc98-81dbbc80b18d" Jun 03 01:43:07.561327 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:07.561292 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:43:07.561805 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:07.561296 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwd6s" Jun 03 01:43:11.584883 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.584848 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:43:11.585357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.584913 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:43:11.587359 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.587333 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74c31246-3616-41a4-b95a-0a5242f1fa28-metrics-tls\") pod \"dns-default-bwd6s\" (UID: \"74c31246-3616-41a4-b95a-0a5242f1fa28\") " pod="openshift-dns/dns-default-bwd6s" Jun 03 01:43:11.587481 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.587463 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc11711b-16fb-4148-bc98-81dbbc80b18d-cert\") pod \"ingress-canary-x4r47\" (UID: \"cc11711b-16fb-4148-bc98-81dbbc80b18d\") " pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:43:11.765437 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.765407 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2xcbn\"" Jun 03 01:43:11.766347 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.766329 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tsbnv\"" Jun 03 01:43:11.773571 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.773554 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4r47" Jun 03 01:43:11.773639 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.773623 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwd6s" Jun 03 01:43:11.920133 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.920110 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bwd6s"] Jun 03 01:43:11.922958 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:43:11.922918 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c31246_3616_41a4_b95a_0a5242f1fa28.slice/crio-5cae9494066b5fa7a3ecb1db41b00c5fe3d3414a970bd122c43edc4e8e782b93 WatchSource:0}: Error finding container 5cae9494066b5fa7a3ecb1db41b00c5fe3d3414a970bd122c43edc4e8e782b93: Status 404 returned error can't find the container with id 5cae9494066b5fa7a3ecb1db41b00c5fe3d3414a970bd122c43edc4e8e782b93 Jun 03 01:43:11.938213 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:43:11.938184 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc11711b_16fb_4148_bc98_81dbbc80b18d.slice/crio-6eec59b14c734d0b75c33094e9441f688bb797c6eb4204f9671719b76e12f3ef WatchSource:0}: Error finding container 6eec59b14c734d0b75c33094e9441f688bb797c6eb4204f9671719b76e12f3ef: Status 404 returned error can't find the container with id 6eec59b14c734d0b75c33094e9441f688bb797c6eb4204f9671719b76e12f3ef Jun 03 01:43:11.939417 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:11.939394 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x4r47"] Jun 03 01:43:12.577830 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:12.577783 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x4r47" event={"ID":"cc11711b-16fb-4148-bc98-81dbbc80b18d","Type":"ContainerStarted","Data":"6eec59b14c734d0b75c33094e9441f688bb797c6eb4204f9671719b76e12f3ef"} Jun 03 01:43:12.579057 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:12.579025 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwd6s" event={"ID":"74c31246-3616-41a4-b95a-0a5242f1fa28","Type":"ContainerStarted","Data":"5cae9494066b5fa7a3ecb1db41b00c5fe3d3414a970bd122c43edc4e8e782b93"} Jun 03 01:43:14.266196 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.266167 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:43:14.266196 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.266201 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:43:14.587243 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.587209 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x4r47" event={"ID":"cc11711b-16fb-4148-bc98-81dbbc80b18d","Type":"ContainerStarted","Data":"ee652114837ed5add966505ca52ab8c5e782de140cde6ade6708bf60e4d6bd81"} Jun 03 01:43:14.588785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.588755 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwd6s" event={"ID":"74c31246-3616-41a4-b95a-0a5242f1fa28","Type":"ContainerStarted","Data":"163819e79411b71b6bac7d3266aac4ff93774cba5f6bb003d0c7352d3ed30d8b"} Jun 03 01:43:14.588897 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.588789 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwd6s" event={"ID":"74c31246-3616-41a4-b95a-0a5242f1fa28","Type":"ContainerStarted","Data":"40847ede4ab99ec7b5fdf7b331db4bae497092aea3737ac7fb45f4474c9e32dc"} Jun 03 01:43:14.588897 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.588888 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bwd6s" Jun 03 01:43:14.604731 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.604679 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x4r47" podStartSLOduration=129.814241329 podStartE2EDuration="2m11.604665052s" podCreationTimestamp="2026-06-03 01:41:03 +0000 UTC" firstStartedPulling="2026-06-03 01:43:11.940338379 +0000 UTC m=+162.594981638" lastFinishedPulling="2026-06-03 01:43:13.730762088 +0000 UTC m=+164.385405361" observedRunningTime="2026-06-03 01:43:14.604135518 +0000 UTC m=+165.258778800" watchObservedRunningTime="2026-06-03 01:43:14.604665052 +0000 UTC m=+165.259308331" Jun 03 01:43:14.620998 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:14.620946 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-bwd6s" podStartSLOduration=129.816293542 podStartE2EDuration="2m11.620932316s" podCreationTimestamp="2026-06-03 01:41:03 +0000 UTC" firstStartedPulling="2026-06-03 01:43:11.924868006 +0000 UTC m=+162.579511264" lastFinishedPulling="2026-06-03 01:43:13.729506779 +0000 UTC m=+164.384150038" observedRunningTime="2026-06-03 01:43:14.619222358 +0000 UTC m=+165.273865640" watchObservedRunningTime="2026-06-03 01:43:14.620932316 +0000 UTC m=+165.275575627" Jun 03 01:43:24.594443 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:24.594401 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bwd6s" Jun 03 01:43:34.271022 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:34.270987 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:43:34.275000 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:34.274970 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5fb755fc7-zjx28" Jun 03 01:43:41.671418 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:41.671378 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0" containerID="19cb3ce941d3c0f577d05c24ec94bde992062d8e2562bd6158505cf8b095d2e7" exitCode=0 Jun 03 01:43:41.671849 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:41.671420 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" event={"ID":"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0","Type":"ContainerDied","Data":"19cb3ce941d3c0f577d05c24ec94bde992062d8e2562bd6158505cf8b095d2e7"} Jun 03 01:43:41.671849 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:41.671782 2553 scope.go:117] "RemoveContainer" containerID="19cb3ce941d3c0f577d05c24ec94bde992062d8e2562bd6158505cf8b095d2e7" Jun 03 01:43:42.676206 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:42.676165 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-tqsx7" event={"ID":"8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0","Type":"ContainerStarted","Data":"0a0036b03a44d15b51a972ead34dcfe0cb01c800cb86a04e2476a1f8f7037ee2"} Jun 03 01:43:56.154034 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:56.153990 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:43:56.175594 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:56.175567 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:43:56.737267 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:43:56.737239 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.100185 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.100145 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:44:14.101457 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.101351 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy" containerID="cri-o://839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" gracePeriod=600 Jun 03 01:44:14.101457 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.101370 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="thanos-sidecar" containerID="cri-o://7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" gracePeriod=600 Jun 03 01:44:14.101457 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.101403 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" gracePeriod=600 Jun 03 01:44:14.101457 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.101401 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-web" containerID="cri-o://f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" gracePeriod=600 Jun 03 01:44:14.101772 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.101382 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="prometheus" containerID="cri-o://ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" gracePeriod=600 Jun 03 01:44:14.101772 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.101591 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="config-reloader" containerID="cri-o://43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" gracePeriod=600 Jun 03 01:44:14.375209 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.375182 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.533397 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533361 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-trusted-ca-bundle\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533397 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533401 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-kube-rbac-proxy\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533422 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533454 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-metrics-client-ca\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533470 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-rulefiles-0\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533489 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533684 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533581 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-metrics-client-certs\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533610 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-web-config\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533743 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-grpc-tls\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533767 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-tls\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533800 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-serving-certs-ca-bundle\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533830 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-config-out\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533858 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-config\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533848 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533895 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-tls-assets\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.533919 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533893 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:44:14.534364 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533942 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-db\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.534364 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.533966 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-thanos-prometheus-http-client-file\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.534364 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.534027 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-kubelet-serving-ca-bundle\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.534364 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.534066 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7m6d\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-kube-api-access-b7m6d\") pod \"8ef28bab-85f7-4094-a9b7-f4458222964c\" (UID: \"8ef28bab-85f7-4094-a9b7-f4458222964c\") " Jun 03 01:44:14.534734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.534384 2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.534734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.534404 2553 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-metrics-client-ca\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.535782 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.535171 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:44:14.536082 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.536058 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:44:14.537246 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.537217 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 01:44:14.537440 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.537415 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 01:44:14.537786 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.537747 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.538655 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.538587 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.538655 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.538592 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-kube-api-access-b7m6d" (OuterVolumeSpecName: "kube-api-access-b7m6d") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "kube-api-access-b7m6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:44:14.538843 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.538667 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.538843 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.538710 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.538843 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.538715 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-config-out" (OuterVolumeSpecName: "config-out") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 01:44:14.538843 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.538764 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.539104 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.539080 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-config" (OuterVolumeSpecName: "config") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.539392 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.539354 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.539492 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.539445 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.540241 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.540218 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:44:14.549427 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.549398 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-web-config" (OuterVolumeSpecName: "web-config") pod "8ef28bab-85f7-4094-a9b7-f4458222964c" (UID: "8ef28bab-85f7-4094-a9b7-f4458222964c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634867 2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634915 2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634926 2553 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-metrics-client-certs\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634939 2553 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-web-config\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634948 2553 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-grpc-tls\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634959 2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634968 2553 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.634969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634979 2553 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-config-out\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.634989 2553 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-config\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635000 2553 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-tls-assets\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635008 2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ef28bab-85f7-4094-a9b7-f4458222964c-prometheus-k8s-db\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635017 2553 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635027 2553 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef28bab-85f7-4094-a9b7-f4458222964c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635036 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7m6d\" (UniqueName: \"kubernetes.io/projected/8ef28bab-85f7-4094-a9b7-f4458222964c-kube-api-access-b7m6d\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635045 2553 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-kube-rbac-proxy\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.635321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.635056 2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ef28bab-85f7-4094-a9b7-f4458222964c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:44:14.777007 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.776975 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" exitCode=0 Jun 03 01:44:14.777007 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777000 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" exitCode=0 Jun 03 01:44:14.777007 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777006 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" exitCode=0 Jun 03 01:44:14.777007 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777014 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" exitCode=0 Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777019 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" exitCode=0 Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777024 2553 generic.go:358] "Generic (PLEG): container finished" podID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" exitCode=0 Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777039 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777076 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777086 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777095 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777103 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777111 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777118 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ef28bab-85f7-4094-a9b7-f4458222964c","Type":"ContainerDied","Data":"62e8162e2f8ea9090ebccf2b2fee2b239ee6d2f08f0f593bb2b263e6c6863a15"} Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777133 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.777238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.777152 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.789048 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.789018 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.796695 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.796673 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.804573 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.804517 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.804668 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.804623 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:44:14.808845 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.808822 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:44:14.813704 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.813679 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.821283 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.821257 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.829497 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.829476 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.834476 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.834448 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:44:14.835012 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.834988 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="prometheus" Jun 03 01:44:14.835012 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835013 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="prometheus" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835030 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-thanos" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835038 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-thanos" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835049 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835056 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835074 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="thanos-sidecar" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835082 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="thanos-sidecar" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835093 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-web" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835101 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-web" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835114 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="config-reloader" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835122 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="config-reloader" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835139 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="init-config-reloader" Jun 03 01:44:14.835192 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835147 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="init-config-reloader" Jun 03 01:44:14.835785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835230 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="thanos-sidecar" Jun 03 01:44:14.835785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835245 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy" Jun 03 01:44:14.835785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835256 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="config-reloader" Jun 03 01:44:14.835785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835270 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="prometheus" Jun 03 01:44:14.835785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835283 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-thanos" Jun 03 01:44:14.835785 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.835292 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" containerName="kube-rbac-proxy-web" Jun 03 01:44:14.838379 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.838361 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.838712 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.838689 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.838760 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.838722 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} err="failed to get container status \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" Jun 03 01:44:14.838760 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.838756 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.839014 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.838997 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.839056 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839021 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} err="failed to get container status \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" Jun 03 01:44:14.839056 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839036 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.839211 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.839197 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.839251 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839214 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} err="failed to get container status \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" Jun 03 01:44:14.839251 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839226 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.839426 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.839403 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.839468 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839432 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} err="failed to get container status \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" Jun 03 01:44:14.839468 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839446 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.839723 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.839705 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.839778 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839726 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} err="failed to get container status \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" Jun 03 01:44:14.839778 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.839742 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.840001 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.839975 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.840039 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.840005 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} err="failed to get container status \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" Jun 03 01:44:14.840039 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.840018 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.840266 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:44:14.840246 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.840325 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.840272 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} err="failed to get container status \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" Jun 03 01:44:14.840325 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.840290 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.840798 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.840705 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} err="failed to get container status \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" Jun 03 01:44:14.840798 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.840731 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.841061 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841041 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} err="failed to get container status \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" Jun 03 01:44:14.841100 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841061 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.841100 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841068 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.841380 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841343 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} err="failed to get container status \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" Jun 03 01:44:14.841380 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841367 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.841648 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841613 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} err="failed to get container status \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" Jun 03 01:44:14.841648 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841647 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.841960 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841941 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} err="failed to get container status \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" Jun 03 01:44:14.841960 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.841960 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.842177 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842163 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} err="failed to get container status \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" Jun 03 01:44:14.842217 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842177 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.842380 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842365 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} err="failed to get container status \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" Jun 03 01:44:14.842380 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842379 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.842596 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842574 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} err="failed to get container status \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" Jun 03 01:44:14.842596 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842595 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.842856 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842835 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} err="failed to get container status \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" Jun 03 01:44:14.842912 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.842857 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.843084 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843069 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} err="failed to get container status \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" Jun 03 01:44:14.843135 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843084 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.843300 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843279 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} err="failed to get container status \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" Jun 03 01:44:14.843353 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843301 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.843547 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843512 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} err="failed to get container status \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" Jun 03 01:44:14.843625 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843554 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.843761 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843741 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} err="failed to get container status \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" Jun 03 01:44:14.843814 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843762 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.843992 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843974 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} err="failed to get container status \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" Jun 03 01:44:14.844059 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.843994 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.844240 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844224 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} err="failed to get container status \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" Jun 03 01:44:14.844284 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844241 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.844439 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844422 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} err="failed to get container status \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" Jun 03 01:44:14.844543 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844440 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.844645 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844628 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} err="failed to get container status \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" Jun 03 01:44:14.844685 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844646 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.844822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844805 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} err="failed to get container status \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" Jun 03 01:44:14.844861 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.844823 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.845019 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845003 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} err="failed to get container status \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" Jun 03 01:44:14.845088 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845020 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.845301 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845282 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} err="failed to get container status \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" Jun 03 01:44:14.845370 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845301 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.845502 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845482 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 03 01:44:14.845633 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845586 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 03 01:44:14.845633 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845606 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} err="failed to get container status \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" Jun 03 01:44:14.845741 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845633 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.845891 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845869 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} err="failed to get container status \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" Jun 03 01:44:14.845937 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.845893 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.846127 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846109 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} err="failed to get container status \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" Jun 03 01:44:14.846174 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846127 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.846342 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846323 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} err="failed to get container status \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" Jun 03 01:44:14.846391 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846343 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.846572 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846549 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} err="failed to get container status \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" Jun 03 01:44:14.846623 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846574 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.846843 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846806 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} err="failed to get container status \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" Jun 03 01:44:14.846914 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.846843 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.847044 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847024 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} err="failed to get container status \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" Jun 03 01:44:14.847084 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847046 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.847263 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847239 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} err="failed to get container status \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" Jun 03 01:44:14.847303 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847266 2553 scope.go:117] "RemoveContainer" containerID="a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83" Jun 03 01:44:14.847517 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847494 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83"} err="failed to get container status \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": rpc error: code = NotFound desc = could not find container \"a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83\": container with ID starting with a94448819d4571cc17ee9fc4bfde2173f9ef391d26dec118e812ec773320ee83 not found: ID does not exist" Jun 03 01:44:14.847615 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847519 2553 scope.go:117] "RemoveContainer" containerID="839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f" Jun 03 01:44:14.847833 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847814 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f"} err="failed to get container status \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": rpc error: code = NotFound desc = could not find container \"839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f\": container with ID starting with 839c74dbe800a1d52cb4f835ca1eec2e15eb65d64a3fad9226c0996ec8b0c07f not found: ID does not exist" Jun 03 01:44:14.847910 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.847834 2553 scope.go:117] "RemoveContainer" containerID="f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008" Jun 03 01:44:14.848108 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848085 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008"} err="failed to get container status \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": rpc error: code = NotFound desc = could not find container \"f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008\": container with ID starting with f8f16cff2300dc2c8c9a2453d41e4ba658e01cdb9a083a227d066a36ea129008 not found: ID does not exist" Jun 03 01:44:14.848171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848111 2553 scope.go:117] "RemoveContainer" containerID="7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c" Jun 03 01:44:14.848361 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848339 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c"} err="failed to get container status \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": rpc error: code = NotFound desc = could not find container \"7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c\": container with ID starting with 7069e02d85d16390b37a816b404579149837f47ef79a6401a3b7c48f3d29cd1c not found: ID does not exist" Jun 03 01:44:14.848435 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848362 2553 scope.go:117] "RemoveContainer" containerID="43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d" Jun 03 01:44:14.848607 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848583 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d"} err="failed to get container status \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": rpc error: code = NotFound desc = could not find container \"43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d\": container with ID starting with 43429fc492d95c94cc5fa4b3e069b46b251e4e227556e2b9bb643287119de25d not found: ID does not exist" Jun 03 01:44:14.848699 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848609 2553 scope.go:117] "RemoveContainer" containerID="ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc" Jun 03 01:44:14.848979 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848856 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc"} err="failed to get container status \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": rpc error: code = NotFound desc = could not find container \"ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc\": container with ID starting with ad2d3ced22fd1ce7bb9273b4b2ddbd9882df79c6fae77d4a333de200c3034bcc not found: ID does not exist" Jun 03 01:44:14.848979 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.848876 2553 scope.go:117] "RemoveContainer" containerID="16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d" Jun 03 01:44:14.849240 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.849222 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d"} err="failed to get container status \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": rpc error: code = NotFound desc = could not find container \"16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d\": container with ID starting with 16334bc4417afb5cf645973c42c7e6f819a626a62a2090531d584b74f08db55d not found: ID does not exist" Jun 03 01:44:14.849955 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.849937 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 03 01:44:14.851902 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.851883 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 03 01:44:14.852003 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.851883 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 03 01:44:14.852003 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.851908 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 03 01:44:14.852003 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.851910 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 03 01:44:14.852003 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.851966 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bstn5\"" Jun 03 01:44:14.852155 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.851910 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 03 01:44:14.852155 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.852058 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 03 01:44:14.852281 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.852269 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4ptuc8tl6tr3p\"" Jun 03 01:44:14.852313 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.852281 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 03 01:44:14.865992 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.865968 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:44:14.883697 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.883664 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 03 01:44:14.887491 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.887432 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 03 01:44:14.937902 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.937864 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.937902 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.937902 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.937919 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.937980 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938000 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938019 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938053 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938109 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938158 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938204 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938234 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938286 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938357 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938302 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-config\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938358 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938382 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938398 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:14.938662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:14.938413 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzcrd\" (UniqueName: \"kubernetes.io/projected/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-kube-api-access-rzcrd\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039748 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039699 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039748 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039750 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039773 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039789 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039808 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-config\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039835 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.039947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.039851 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040024 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040063 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rzcrd\" (UniqueName: \"kubernetes.io/projected/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-kube-api-access-rzcrd\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040110 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040150 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040177 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040209 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040242 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040605 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040271 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040605 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040312 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040605 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040342 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040605 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040365 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.040605 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.040391 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.041191 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.041166 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.041345 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.041212 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.043383 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.043189 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.043383 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.043189 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.043383 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.043306 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-config\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.043662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.043397 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.043662 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.043579 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.044041 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.043830 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.044297 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.044251 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.044595 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.044569 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.044721 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.044700 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.044898 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.044879 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.045489 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.045471 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.045629 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.045613 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.046281 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.046261 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.046485 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.046469 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.049375 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.049357 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzcrd\" (UniqueName: \"kubernetes.io/projected/cb3b9ce4-4c27-41d8-916a-cec4dcd7e385-kube-api-access-rzcrd\") pod \"prometheus-k8s-0\" (UID: \"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.151863 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.151770 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:44:15.293851 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.293823 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 01:44:15.296194 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:44:15.296168 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3b9ce4_4c27_41d8_916a_cec4dcd7e385.slice/crio-fd0916d13d87119cc297775f8a5c7344c65c038ab1f31e90534e2082843e46ab WatchSource:0}: Error finding container fd0916d13d87119cc297775f8a5c7344c65c038ab1f31e90534e2082843e46ab: Status 404 returned error can't find the container with id fd0916d13d87119cc297775f8a5c7344c65c038ab1f31e90534e2082843e46ab Jun 03 01:44:15.783079 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.783044 2553 generic.go:358] "Generic (PLEG): container finished" podID="cb3b9ce4-4c27-41d8-916a-cec4dcd7e385" containerID="c4b57f76a9f12566b16c57a6fbec8f00a2270a8ce3705443aa90262172eaa28b" exitCode=0 Jun 03 01:44:15.783253 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.783112 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerDied","Data":"c4b57f76a9f12566b16c57a6fbec8f00a2270a8ce3705443aa90262172eaa28b"} Jun 03 01:44:15.783253 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.783130 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"fd0916d13d87119cc297775f8a5c7344c65c038ab1f31e90534e2082843e46ab"} Jun 03 01:44:15.940644 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:15.940607 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ef28bab-85f7-4094-a9b7-f4458222964c" path="/var/lib/kubelet/pods/8ef28bab-85f7-4094-a9b7-f4458222964c/volumes" Jun 03 01:44:16.792239 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.792201 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"9cff74c1020c09116a3b0178c0a9a1d957e5cab08201be8028a070bbcd1b150f"} Jun 03 01:44:16.792239 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.792240 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"00a634b2f2977bda75bb609ea4ba920b71a568e9ad42688fc210935aaeae9c46"} Jun 03 01:44:16.792658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.792250 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"a50fe01db217407782733abf57e4ecb3075075942670dd8eb7b91cf3f8a85787"} Jun 03 01:44:16.792658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.792258 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"a9324264567cb92d8ed18aa8f46ac75360a0fbefad159b7aa3efc99a43faf28c"} Jun 03 01:44:16.792658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.792266 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"6fc532520031809aed459059ccfc89be296b2bc2e88ca76833915ecae1f37182"} Jun 03 01:44:16.792658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.792275 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb3b9ce4-4c27-41d8-916a-cec4dcd7e385","Type":"ContainerStarted","Data":"847e90cd862914a986a85efe03245690c7aed48ccd6d2a20838f18a0ce293f6e"} Jun 03 01:44:16.819758 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:16.819698 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.819677991 podStartE2EDuration="2.819677991s" podCreationTimestamp="2026-06-03 01:44:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:44:16.817638579 +0000 UTC m=+227.472281857" watchObservedRunningTime="2026-06-03 01:44:16.819677991 +0000 UTC m=+227.474321272" Jun 03 01:44:20.152518 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:44:20.152459 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:45:15.152188 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:15.152131 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:45:15.168950 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:15.168921 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:45:15.993707 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:15.993674 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 01:45:29.788564 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:29.788509 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:45:29.794696 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:29.794667 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:45:29.799330 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:29.799302 2553 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 01:45:30.363461 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.363423 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-5smtw"] Jun 03 01:45:30.366516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.366500 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.369096 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.369073 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 03 01:45:30.369196 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.369101 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 03 01:45:30.370134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.370121 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5ts76\"" Jun 03 01:45:30.376411 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.376389 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-5smtw"] Jun 03 01:45:30.499037 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.498997 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd8k9\" (UniqueName: \"kubernetes.io/projected/635057bf-7e19-4960-877b-66c7a8329c50-kube-api-access-gd8k9\") pod \"cert-manager-webhook-587ccfb98-5smtw\" (UID: \"635057bf-7e19-4960-877b-66c7a8329c50\") " pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.499037 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.499044 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/635057bf-7e19-4960-877b-66c7a8329c50-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-5smtw\" (UID: \"635057bf-7e19-4960-877b-66c7a8329c50\") " pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.600502 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.600469 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gd8k9\" (UniqueName: \"kubernetes.io/projected/635057bf-7e19-4960-877b-66c7a8329c50-kube-api-access-gd8k9\") pod \"cert-manager-webhook-587ccfb98-5smtw\" (UID: \"635057bf-7e19-4960-877b-66c7a8329c50\") " pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.600687 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.600511 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/635057bf-7e19-4960-877b-66c7a8329c50-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-5smtw\" (UID: \"635057bf-7e19-4960-877b-66c7a8329c50\") " pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.608427 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.608405 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/635057bf-7e19-4960-877b-66c7a8329c50-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-5smtw\" (UID: \"635057bf-7e19-4960-877b-66c7a8329c50\") " pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.608724 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.608703 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd8k9\" (UniqueName: \"kubernetes.io/projected/635057bf-7e19-4960-877b-66c7a8329c50-kube-api-access-gd8k9\") pod \"cert-manager-webhook-587ccfb98-5smtw\" (UID: \"635057bf-7e19-4960-877b-66c7a8329c50\") " pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.690238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.690151 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:30.820685 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:30.820655 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-5smtw"] Jun 03 01:45:30.823324 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:45:30.823280 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod635057bf_7e19_4960_877b_66c7a8329c50.slice/crio-69b8d47767e39a351e9de03e719a29f969ce72c6ecbb2fc76223a124b7721031 WatchSource:0}: Error finding container 69b8d47767e39a351e9de03e719a29f969ce72c6ecbb2fc76223a124b7721031: Status 404 returned error can't find the container with id 69b8d47767e39a351e9de03e719a29f969ce72c6ecbb2fc76223a124b7721031 Jun 03 01:45:31.018321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:31.018229 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" event={"ID":"635057bf-7e19-4960-877b-66c7a8329c50","Type":"ContainerStarted","Data":"69b8d47767e39a351e9de03e719a29f969ce72c6ecbb2fc76223a124b7721031"} Jun 03 01:45:35.032986 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:35.032950 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" event={"ID":"635057bf-7e19-4960-877b-66c7a8329c50","Type":"ContainerStarted","Data":"1e88237f2ee6424b6fdde1a7c1069c63c45ed7b495da39a642d297d044d93da6"} Jun 03 01:45:35.033390 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:35.033037 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:35.048609 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:35.048558 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" podStartSLOduration=0.970454812 podStartE2EDuration="5.048521307s" podCreationTimestamp="2026-06-03 01:45:30 +0000 UTC" firstStartedPulling="2026-06-03 01:45:30.825208381 +0000 UTC m=+301.479851640" lastFinishedPulling="2026-06-03 01:45:34.90327487 +0000 UTC m=+305.557918135" observedRunningTime="2026-06-03 01:45:35.046818171 +0000 UTC m=+305.701461451" watchObservedRunningTime="2026-06-03 01:45:35.048521307 +0000 UTC m=+305.703164586" Jun 03 01:45:41.037733 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:41.037706 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-5smtw" Jun 03 01:45:49.434257 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.434223 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-drksq"] Jun 03 01:45:49.441269 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.441244 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.443920 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.443892 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nb8ls\"" Jun 03 01:45:49.447897 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.447871 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-drksq"] Jun 03 01:45:49.461255 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.461220 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hf2z\" (UniqueName: \"kubernetes.io/projected/0084db95-dec6-4ad5-8672-8a34db746cfd-kube-api-access-5hf2z\") pod \"cert-manager-79c8d999ff-drksq\" (UID: \"0084db95-dec6-4ad5-8672-8a34db746cfd\") " pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.461255 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.461256 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0084db95-dec6-4ad5-8672-8a34db746cfd-bound-sa-token\") pod \"cert-manager-79c8d999ff-drksq\" (UID: \"0084db95-dec6-4ad5-8672-8a34db746cfd\") " pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.561718 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.561679 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5hf2z\" (UniqueName: \"kubernetes.io/projected/0084db95-dec6-4ad5-8672-8a34db746cfd-kube-api-access-5hf2z\") pod \"cert-manager-79c8d999ff-drksq\" (UID: \"0084db95-dec6-4ad5-8672-8a34db746cfd\") " pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.561718 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.561720 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0084db95-dec6-4ad5-8672-8a34db746cfd-bound-sa-token\") pod \"cert-manager-79c8d999ff-drksq\" (UID: \"0084db95-dec6-4ad5-8672-8a34db746cfd\") " pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.570163 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.570124 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0084db95-dec6-4ad5-8672-8a34db746cfd-bound-sa-token\") pod \"cert-manager-79c8d999ff-drksq\" (UID: \"0084db95-dec6-4ad5-8672-8a34db746cfd\") " pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.570285 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.570234 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hf2z\" (UniqueName: \"kubernetes.io/projected/0084db95-dec6-4ad5-8672-8a34db746cfd-kube-api-access-5hf2z\") pod \"cert-manager-79c8d999ff-drksq\" (UID: \"0084db95-dec6-4ad5-8672-8a34db746cfd\") " pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.751492 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.751403 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-drksq" Jun 03 01:45:49.881032 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.881006 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-drksq"] Jun 03 01:45:49.883492 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:45:49.883458 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0084db95_dec6_4ad5_8672_8a34db746cfd.slice/crio-b35f604caba535531ad85b0180716b3d9ffcb172f3f0f4c7158565068ca3d05e WatchSource:0}: Error finding container b35f604caba535531ad85b0180716b3d9ffcb172f3f0f4c7158565068ca3d05e: Status 404 returned error can't find the container with id b35f604caba535531ad85b0180716b3d9ffcb172f3f0f4c7158565068ca3d05e Jun 03 01:45:49.886147 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:49.886132 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 01:45:50.080128 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:50.080035 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-drksq" event={"ID":"0084db95-dec6-4ad5-8672-8a34db746cfd","Type":"ContainerStarted","Data":"99412eeefe4e522c115bf39b067d8d036f588a166a7ce65a0e5e0e75856eca77"} Jun 03 01:45:50.080128 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:50.080077 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-drksq" event={"ID":"0084db95-dec6-4ad5-8672-8a34db746cfd","Type":"ContainerStarted","Data":"b35f604caba535531ad85b0180716b3d9ffcb172f3f0f4c7158565068ca3d05e"} Jun 03 01:45:50.096130 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:45:50.096084 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-drksq" podStartSLOduration=1.096070342 podStartE2EDuration="1.096070342s" podCreationTimestamp="2026-06-03 01:45:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:45:50.094455366 +0000 UTC m=+320.749098657" watchObservedRunningTime="2026-06-03 01:45:50.096070342 +0000 UTC m=+320.750713621" Jun 03 01:46:05.322381 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.322345 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf"] Jun 03 01:46:05.327673 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.327644 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.335783 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.335757 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 01:46:05.335917 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.335903 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 01:46:05.335982 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.335969 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 01:46:05.336264 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.336244 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6wj9t\"" Jun 03 01:46:05.336362 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.336249 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 01:46:05.341584 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.341559 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf"] Jun 03 01:46:05.395862 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.395822 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.396033 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.395874 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-manifests\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.396033 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.395930 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frplx\" (UniqueName: \"kubernetes.io/projected/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-kube-api-access-frplx\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.396033 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.395966 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.396163 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.396026 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-tmp\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.497565 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.497488 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-manifests\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.497759 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.497585 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frplx\" (UniqueName: \"kubernetes.io/projected/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-kube-api-access-frplx\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.497759 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.497644 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.497759 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.497672 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-tmp\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.497759 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.497731 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.497972 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.497911 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-manifests\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.500140 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.500116 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-tmp\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.500338 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.500316 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.500397 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.500368 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.508793 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.508772 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frplx\" (UniqueName: \"kubernetes.io/projected/cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7-kube-api-access-frplx\") pod \"opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf\" (UID: \"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7\") " pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.638958 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.638921 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:05.774833 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:05.774800 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf"] Jun 03 01:46:05.777485 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:46:05.777448 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae7f9a0_bd6d_46d7_a8c7_87aafc235ed7.slice/crio-275dd4a4b3fe6c0914671a090793ee47889964886caa2a7342ab2a7c9c0cc5fa WatchSource:0}: Error finding container 275dd4a4b3fe6c0914671a090793ee47889964886caa2a7342ab2a7c9c0cc5fa: Status 404 returned error can't find the container with id 275dd4a4b3fe6c0914671a090793ee47889964886caa2a7342ab2a7c9c0cc5fa Jun 03 01:46:06.132644 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:06.132602 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" event={"ID":"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7","Type":"ContainerStarted","Data":"275dd4a4b3fe6c0914671a090793ee47889964886caa2a7342ab2a7c9c0cc5fa"} Jun 03 01:46:09.145416 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:09.145372 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" event={"ID":"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7","Type":"ContainerStarted","Data":"93b2a9c48b116b1c00881bc693345e7e12ff4067883436ac3f2c6c7795866cd8"} Jun 03 01:46:10.150692 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:10.150654 2553 generic.go:358] "Generic (PLEG): container finished" podID="cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7" containerID="93b2a9c48b116b1c00881bc693345e7e12ff4067883436ac3f2c6c7795866cd8" exitCode=0 Jun 03 01:46:10.151084 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:10.150744 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" event={"ID":"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7","Type":"ContainerDied","Data":"93b2a9c48b116b1c00881bc693345e7e12ff4067883436ac3f2c6c7795866cd8"} Jun 03 01:46:11.156253 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:11.156215 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" event={"ID":"cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7","Type":"ContainerStarted","Data":"cd4a398bdba0f55357940113b9da1fd59cea734c6e67a7e8cb4a5f0a50682d8b"} Jun 03 01:46:11.156809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:11.156325 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:11.178428 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:11.178370 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" podStartSLOduration=1.465426747 podStartE2EDuration="6.178355031s" podCreationTimestamp="2026-06-03 01:46:05 +0000 UTC" firstStartedPulling="2026-06-03 01:46:05.779272456 +0000 UTC m=+336.433915714" lastFinishedPulling="2026-06-03 01:46:10.492200737 +0000 UTC m=+341.146843998" observedRunningTime="2026-06-03 01:46:11.17622856 +0000 UTC m=+341.830871866" watchObservedRunningTime="2026-06-03 01:46:11.178355031 +0000 UTC m=+341.832998312" Jun 03 01:46:22.161593 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:22.161497 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf" Jun 03 01:46:23.844574 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.844514 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp"] Jun 03 01:46:23.940389 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.940351 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp"] Jun 03 01:46:23.940640 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.940491 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:23.943661 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.943633 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 03 01:46:23.944368 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.944346 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 03 01:46:23.944832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.944806 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-npgp5\"" Jun 03 01:46:23.944937 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.944882 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 03 01:46:23.944998 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.944806 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 03 01:46:23.945054 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:23.944884 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 03 01:46:24.064494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.064461 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpmk\" (UniqueName: \"kubernetes.io/projected/a798cf0d-3139-446f-91c0-2b8763a10266-kube-api-access-qhpmk\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.064494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.064508 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a798cf0d-3139-446f-91c0-2b8763a10266-cert\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.064737 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.064563 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a798cf0d-3139-446f-91c0-2b8763a10266-metrics-cert\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.064737 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.064623 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a798cf0d-3139-446f-91c0-2b8763a10266-manager-config\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.165995 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.165916 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpmk\" (UniqueName: \"kubernetes.io/projected/a798cf0d-3139-446f-91c0-2b8763a10266-kube-api-access-qhpmk\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.165995 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.165958 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a798cf0d-3139-446f-91c0-2b8763a10266-cert\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.166216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.166079 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a798cf0d-3139-446f-91c0-2b8763a10266-metrics-cert\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.166216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.166145 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a798cf0d-3139-446f-91c0-2b8763a10266-manager-config\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.166817 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.166776 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a798cf0d-3139-446f-91c0-2b8763a10266-manager-config\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.168627 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.168598 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a798cf0d-3139-446f-91c0-2b8763a10266-cert\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.168794 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.168773 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a798cf0d-3139-446f-91c0-2b8763a10266-metrics-cert\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.204170 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.204131 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpmk\" (UniqueName: \"kubernetes.io/projected/a798cf0d-3139-446f-91c0-2b8763a10266-kube-api-access-qhpmk\") pod \"lws-controller-manager-7bc87d8b78-q46fp\" (UID: \"a798cf0d-3139-446f-91c0-2b8763a10266\") " pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.251349 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.251311 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:24.404155 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:24.404112 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp"] Jun 03 01:46:24.407549 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:46:24.407504 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda798cf0d_3139_446f_91c0_2b8763a10266.slice/crio-30d87d710f2037756b69575eadb98c27c28021b1a7af94fb39b8d0570c30e670 WatchSource:0}: Error finding container 30d87d710f2037756b69575eadb98c27c28021b1a7af94fb39b8d0570c30e670: Status 404 returned error can't find the container with id 30d87d710f2037756b69575eadb98c27c28021b1a7af94fb39b8d0570c30e670 Jun 03 01:46:25.206050 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:25.206004 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" event={"ID":"a798cf0d-3139-446f-91c0-2b8763a10266","Type":"ContainerStarted","Data":"30d87d710f2037756b69575eadb98c27c28021b1a7af94fb39b8d0570c30e670"} Jun 03 01:46:27.216011 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:27.215914 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" event={"ID":"a798cf0d-3139-446f-91c0-2b8763a10266","Type":"ContainerStarted","Data":"e67e04cd27189a4c021d4c52f6e31ea2f24fafcfb0accb75fd29b64b33c6037d"} Jun 03 01:46:27.216438 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:27.216038 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:46:27.233904 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:27.233845 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" podStartSLOduration=1.680239805 podStartE2EDuration="4.233829525s" podCreationTimestamp="2026-06-03 01:46:23 +0000 UTC" firstStartedPulling="2026-06-03 01:46:24.409420677 +0000 UTC m=+355.064063935" lastFinishedPulling="2026-06-03 01:46:26.963010396 +0000 UTC m=+357.617653655" observedRunningTime="2026-06-03 01:46:27.231918417 +0000 UTC m=+357.886561689" watchObservedRunningTime="2026-06-03 01:46:27.233829525 +0000 UTC m=+357.888472783" Jun 03 01:46:38.223134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:46:38.223099 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bc87d8b78-q46fp" Jun 03 01:47:00.040418 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.040383 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf"] Jun 03 01:47:00.043991 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.043965 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.046719 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.046696 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 01:47:00.046946 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.046737 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 01:47:00.047063 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.046749 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 01:47:00.047063 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.046753 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-84mxf\"" Jun 03 01:47:00.055190 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.055165 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf"] Jun 03 01:47:00.089610 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089558 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.089816 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089635 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.089816 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089653 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.089816 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089670 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z965q\" (UniqueName: \"kubernetes.io/projected/18f717c7-f71f-451f-8e35-68cf623ff703-kube-api-access-z965q\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.089816 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089694 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f717c7-f71f-451f-8e35-68cf623ff703-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.089816 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089772 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.089816 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089801 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.090033 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089829 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f717c7-f71f-451f-8e35-68cf623ff703-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.090033 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.089857 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f717c7-f71f-451f-8e35-68cf623ff703-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.190637 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190593 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.190832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190656 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.190832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190696 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f717c7-f71f-451f-8e35-68cf623ff703-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.190832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190730 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f717c7-f71f-451f-8e35-68cf623ff703-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.190832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190763 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.190832 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190803 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191069 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190923 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191069 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.190987 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z965q\" (UniqueName: \"kubernetes.io/projected/18f717c7-f71f-451f-8e35-68cf623ff703-kube-api-access-z965q\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191069 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.191030 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f717c7-f71f-451f-8e35-68cf623ff703-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191069 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.191035 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191269 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.191075 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191269 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.191157 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191514 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.191494 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.191653 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.191635 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f717c7-f71f-451f-8e35-68cf623ff703-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.193297 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.193278 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f717c7-f71f-451f-8e35-68cf623ff703-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.193700 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.193679 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f717c7-f71f-451f-8e35-68cf623ff703-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.199969 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.199945 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f717c7-f71f-451f-8e35-68cf623ff703-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.200216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.200198 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z965q\" (UniqueName: \"kubernetes.io/projected/18f717c7-f71f-451f-8e35-68cf623ff703-kube-api-access-z965q\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8qhjnf\" (UID: \"18f717c7-f71f-451f-8e35-68cf623ff703\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.356439 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.356403 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:00.497238 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.497211 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf"] Jun 03 01:47:00.499999 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:47:00.499968 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f717c7_f71f_451f_8e35_68cf623ff703.slice/crio-c314bd1f914d05fdc2f0af909da506bb356c970ad66ecd6bbbc3c7c7b5f332bb WatchSource:0}: Error finding container c314bd1f914d05fdc2f0af909da506bb356c970ad66ecd6bbbc3c7c7b5f332bb: Status 404 returned error can't find the container with id c314bd1f914d05fdc2f0af909da506bb356c970ad66ecd6bbbc3c7c7b5f332bb Jun 03 01:47:00.886661 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.886627 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7b94c665b4-n5hkv"] Jun 03 01:47:00.895236 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.895201 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:00.897638 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.897608 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7b94c665b4-n5hkv"] Jun 03 01:47:00.898012 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.897989 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-55gst\"" Jun 03 01:47:00.898258 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.898241 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 03 01:47:00.997795 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.997763 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f7z7\" (UniqueName: \"kubernetes.io/projected/28135a58-5e1b-49ad-aa8d-09c8296676ce-kube-api-access-8f7z7\") pod \"dashboard-redirect-7b94c665b4-n5hkv\" (UID: \"28135a58-5e1b-49ad-aa8d-09c8296676ce\") " pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:00.997961 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:00.997827 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/28135a58-5e1b-49ad-aa8d-09c8296676ce-redirect-config\") pod \"dashboard-redirect-7b94c665b4-n5hkv\" (UID: \"28135a58-5e1b-49ad-aa8d-09c8296676ce\") " pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:01.098618 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.098579 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/28135a58-5e1b-49ad-aa8d-09c8296676ce-redirect-config\") pod \"dashboard-redirect-7b94c665b4-n5hkv\" (UID: \"28135a58-5e1b-49ad-aa8d-09c8296676ce\") " pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:01.099078 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.098687 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8f7z7\" (UniqueName: \"kubernetes.io/projected/28135a58-5e1b-49ad-aa8d-09c8296676ce-kube-api-access-8f7z7\") pod \"dashboard-redirect-7b94c665b4-n5hkv\" (UID: \"28135a58-5e1b-49ad-aa8d-09c8296676ce\") " pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:01.099276 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.099253 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/28135a58-5e1b-49ad-aa8d-09c8296676ce-redirect-config\") pod \"dashboard-redirect-7b94c665b4-n5hkv\" (UID: \"28135a58-5e1b-49ad-aa8d-09c8296676ce\") " pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:01.106306 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.106280 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f7z7\" (UniqueName: \"kubernetes.io/projected/28135a58-5e1b-49ad-aa8d-09c8296676ce-kube-api-access-8f7z7\") pod \"dashboard-redirect-7b94c665b4-n5hkv\" (UID: \"28135a58-5e1b-49ad-aa8d-09c8296676ce\") " pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:01.208216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.208127 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" Jun 03 01:47:01.342956 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.342931 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7b94c665b4-n5hkv"] Jun 03 01:47:01.344751 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:01.344720 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" event={"ID":"18f717c7-f71f-451f-8e35-68cf623ff703","Type":"ContainerStarted","Data":"c314bd1f914d05fdc2f0af909da506bb356c970ad66ecd6bbbc3c7c7b5f332bb"} Jun 03 01:47:01.345758 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:47:01.345734 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28135a58_5e1b_49ad_aa8d_09c8296676ce.slice/crio-4228d9ea8fb7732ad4a07b600346654857283de848426471f899960f029c26be WatchSource:0}: Error finding container 4228d9ea8fb7732ad4a07b600346654857283de848426471f899960f029c26be: Status 404 returned error can't find the container with id 4228d9ea8fb7732ad4a07b600346654857283de848426471f899960f029c26be Jun 03 01:47:02.349084 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:02.349044 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" event={"ID":"28135a58-5e1b-49ad-aa8d-09c8296676ce","Type":"ContainerStarted","Data":"4228d9ea8fb7732ad4a07b600346654857283de848426471f899960f029c26be"} Jun 03 01:47:18.730853 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:18.730812 2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 03 01:47:18.731221 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:18.730883 2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 03 01:47:18.731221 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:18.730912 2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 03 01:47:19.412206 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:19.412164 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" event={"ID":"28135a58-5e1b-49ad-aa8d-09c8296676ce","Type":"ContainerStarted","Data":"093152d36d12c1e723bf037f391251d4a17c8f9af84bd0f530a7079f4a6baea5"} Jun 03 01:47:19.413477 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:19.413452 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" event={"ID":"18f717c7-f71f-451f-8e35-68cf623ff703","Type":"ContainerStarted","Data":"283bb435b7def4c4f628db7e20461804aec913ec094494a6ceef618a39062f53"} Jun 03 01:47:19.427942 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:19.427898 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7b94c665b4-n5hkv" podStartSLOduration=2.049667915 podStartE2EDuration="19.427884526s" podCreationTimestamp="2026-06-03 01:47:00 +0000 UTC" firstStartedPulling="2026-06-03 01:47:01.346883624 +0000 UTC m=+392.001526882" lastFinishedPulling="2026-06-03 01:47:18.725100228 +0000 UTC m=+409.379743493" observedRunningTime="2026-06-03 01:47:19.42638613 +0000 UTC m=+410.081029413" watchObservedRunningTime="2026-06-03 01:47:19.427884526 +0000 UTC m=+410.082527806" Jun 03 01:47:19.446186 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:19.446118 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" podStartSLOduration=1.21755857 podStartE2EDuration="19.446095066s" podCreationTimestamp="2026-06-03 01:47:00 +0000 UTC" firstStartedPulling="2026-06-03 01:47:00.50196566 +0000 UTC m=+391.156608918" lastFinishedPulling="2026-06-03 01:47:18.730502153 +0000 UTC m=+409.385145414" observedRunningTime="2026-06-03 01:47:19.443084963 +0000 UTC m=+410.097728244" watchObservedRunningTime="2026-06-03 01:47:19.446095066 +0000 UTC m=+410.100738348" Jun 03 01:47:20.356628 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:20.356587 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:20.361268 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:20.361243 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:20.417749 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:20.417711 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:20.418741 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:20.418700 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8qhjnf" Jun 03 01:47:25.926254 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.926215 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f7wp9"] Jun 03 01:47:25.929484 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.929453 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:25.932086 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.932050 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 01:47:25.932201 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.932090 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 01:47:25.933137 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.933120 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mz726\"" Jun 03 01:47:25.934991 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.934965 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpnfw\" (UniqueName: \"kubernetes.io/projected/26a855e1-0a2d-4658-a703-3d0a6d8404e5-kube-api-access-fpnfw\") pod \"kuadrant-operator-catalog-f7wp9\" (UID: \"26a855e1-0a2d-4658-a703-3d0a6d8404e5\") " pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:25.940575 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:25.940552 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f7wp9"] Jun 03 01:47:26.036395 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.036351 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fpnfw\" (UniqueName: \"kubernetes.io/projected/26a855e1-0a2d-4658-a703-3d0a6d8404e5-kube-api-access-fpnfw\") pod \"kuadrant-operator-catalog-f7wp9\" (UID: \"26a855e1-0a2d-4658-a703-3d0a6d8404e5\") " pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:26.044327 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.044289 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpnfw\" (UniqueName: \"kubernetes.io/projected/26a855e1-0a2d-4658-a703-3d0a6d8404e5-kube-api-access-fpnfw\") pod \"kuadrant-operator-catalog-f7wp9\" (UID: \"26a855e1-0a2d-4658-a703-3d0a6d8404e5\") " pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:26.241297 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.241202 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:26.301961 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.301927 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f7wp9"] Jun 03 01:47:26.393330 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.393248 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f7wp9"] Jun 03 01:47:26.396212 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:47:26.396181 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a855e1_0a2d_4658_a703_3d0a6d8404e5.slice/crio-fad574d0f380ce6b907b6e90f01fff7603f3149e23df3b01745fdb64c505c4df WatchSource:0}: Error finding container fad574d0f380ce6b907b6e90f01fff7603f3149e23df3b01745fdb64c505c4df: Status 404 returned error can't find the container with id fad574d0f380ce6b907b6e90f01fff7603f3149e23df3b01745fdb64c505c4df Jun 03 01:47:26.438494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.438456 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" event={"ID":"26a855e1-0a2d-4658-a703-3d0a6d8404e5","Type":"ContainerStarted","Data":"fad574d0f380ce6b907b6e90f01fff7603f3149e23df3b01745fdb64c505c4df"} Jun 03 01:47:26.512239 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.512152 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmlwd"] Jun 03 01:47:26.515623 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.515602 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:26.522656 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.522621 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmlwd"] Jun 03 01:47:26.540304 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.540270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpsqd\" (UniqueName: \"kubernetes.io/projected/d90d35e8-e43d-4d8c-ad85-2808dfcb7135-kube-api-access-qpsqd\") pod \"kuadrant-operator-catalog-mmlwd\" (UID: \"d90d35e8-e43d-4d8c-ad85-2808dfcb7135\") " pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:26.641524 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.641486 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qpsqd\" (UniqueName: \"kubernetes.io/projected/d90d35e8-e43d-4d8c-ad85-2808dfcb7135-kube-api-access-qpsqd\") pod \"kuadrant-operator-catalog-mmlwd\" (UID: \"d90d35e8-e43d-4d8c-ad85-2808dfcb7135\") " pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:26.649977 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.649943 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpsqd\" (UniqueName: \"kubernetes.io/projected/d90d35e8-e43d-4d8c-ad85-2808dfcb7135-kube-api-access-qpsqd\") pod \"kuadrant-operator-catalog-mmlwd\" (UID: \"d90d35e8-e43d-4d8c-ad85-2808dfcb7135\") " pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:26.826428 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.826328 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:26.965099 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:26.965071 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmlwd"] Jun 03 01:47:26.966455 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:47:26.966430 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90d35e8_e43d_4d8c_ad85_2808dfcb7135.slice/crio-50ea76ddbf5f26d77a9140d42bb55afac72c63b73aa5a35392160740adb46f12 WatchSource:0}: Error finding container 50ea76ddbf5f26d77a9140d42bb55afac72c63b73aa5a35392160740adb46f12: Status 404 returned error can't find the container with id 50ea76ddbf5f26d77a9140d42bb55afac72c63b73aa5a35392160740adb46f12 Jun 03 01:47:27.443776 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:27.443735 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" event={"ID":"d90d35e8-e43d-4d8c-ad85-2808dfcb7135","Type":"ContainerStarted","Data":"50ea76ddbf5f26d77a9140d42bb55afac72c63b73aa5a35392160740adb46f12"} Jun 03 01:47:29.452928 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.452891 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" event={"ID":"26a855e1-0a2d-4658-a703-3d0a6d8404e5","Type":"ContainerStarted","Data":"359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce"} Jun 03 01:47:29.453373 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.452989 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" podUID="26a855e1-0a2d-4658-a703-3d0a6d8404e5" containerName="registry-server" containerID="cri-o://359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce" gracePeriod=2 Jun 03 01:47:29.454382 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.454355 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" event={"ID":"d90d35e8-e43d-4d8c-ad85-2808dfcb7135","Type":"ContainerStarted","Data":"89b4e82047670bd973319852ebb5cadd381e71144dd0b4714c5248928d5e9b6a"} Jun 03 01:47:29.469440 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.469230 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" podStartSLOduration=2.201720974 podStartE2EDuration="4.469212323s" podCreationTimestamp="2026-06-03 01:47:25 +0000 UTC" firstStartedPulling="2026-06-03 01:47:26.397517176 +0000 UTC m=+417.052160433" lastFinishedPulling="2026-06-03 01:47:28.665008521 +0000 UTC m=+419.319651782" observedRunningTime="2026-06-03 01:47:29.468559382 +0000 UTC m=+420.123202661" watchObservedRunningTime="2026-06-03 01:47:29.469212323 +0000 UTC m=+420.123855606" Jun 03 01:47:29.486092 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.486034 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" podStartSLOduration=1.7861011919999998 podStartE2EDuration="3.486018136s" podCreationTimestamp="2026-06-03 01:47:26 +0000 UTC" firstStartedPulling="2026-06-03 01:47:26.96788258 +0000 UTC m=+417.622525839" lastFinishedPulling="2026-06-03 01:47:28.667799526 +0000 UTC m=+419.322442783" observedRunningTime="2026-06-03 01:47:29.485663949 +0000 UTC m=+420.140307228" watchObservedRunningTime="2026-06-03 01:47:29.486018136 +0000 UTC m=+420.140661415" Jun 03 01:47:29.693867 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.693845 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:29.772194 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.772107 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpnfw\" (UniqueName: \"kubernetes.io/projected/26a855e1-0a2d-4658-a703-3d0a6d8404e5-kube-api-access-fpnfw\") pod \"26a855e1-0a2d-4658-a703-3d0a6d8404e5\" (UID: \"26a855e1-0a2d-4658-a703-3d0a6d8404e5\") " Jun 03 01:47:29.774389 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.774357 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a855e1-0a2d-4658-a703-3d0a6d8404e5-kube-api-access-fpnfw" (OuterVolumeSpecName: "kube-api-access-fpnfw") pod "26a855e1-0a2d-4658-a703-3d0a6d8404e5" (UID: "26a855e1-0a2d-4658-a703-3d0a6d8404e5"). InnerVolumeSpecName "kube-api-access-fpnfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:47:29.872917 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:29.872878 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpnfw\" (UniqueName: \"kubernetes.io/projected/26a855e1-0a2d-4658-a703-3d0a6d8404e5-kube-api-access-fpnfw\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:47:30.459111 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.459075 2553 generic.go:358] "Generic (PLEG): container finished" podID="26a855e1-0a2d-4658-a703-3d0a6d8404e5" containerID="359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce" exitCode=0 Jun 03 01:47:30.459660 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.459133 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" Jun 03 01:47:30.459660 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.459168 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" event={"ID":"26a855e1-0a2d-4658-a703-3d0a6d8404e5","Type":"ContainerDied","Data":"359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce"} Jun 03 01:47:30.459660 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.459215 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f7wp9" event={"ID":"26a855e1-0a2d-4658-a703-3d0a6d8404e5","Type":"ContainerDied","Data":"fad574d0f380ce6b907b6e90f01fff7603f3149e23df3b01745fdb64c505c4df"} Jun 03 01:47:30.459660 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.459239 2553 scope.go:117] "RemoveContainer" containerID="359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce" Jun 03 01:47:30.468404 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.468387 2553 scope.go:117] "RemoveContainer" containerID="359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce" Jun 03 01:47:30.468709 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:47:30.468683 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce\": container with ID starting with 359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce not found: ID does not exist" containerID="359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce" Jun 03 01:47:30.468815 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.468716 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce"} err="failed to get container status \"359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce\": rpc error: code = NotFound desc = could not find container \"359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce\": container with ID starting with 359753a847fc14d07eb3a1bec15efe574f2c2673e020ddc1dcf7a0966e802cce not found: ID does not exist" Jun 03 01:47:30.475967 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.475941 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f7wp9"] Jun 03 01:47:30.479119 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:30.479097 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f7wp9"] Jun 03 01:47:31.940159 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:31.940123 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a855e1-0a2d-4658-a703-3d0a6d8404e5" path="/var/lib/kubelet/pods/26a855e1-0a2d-4658-a703-3d0a6d8404e5/volumes" Jun 03 01:47:36.827256 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:36.827217 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:36.827256 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:36.827262 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:36.850677 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:36.850640 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:37.514098 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:37.514071 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-mmlwd" Jun 03 01:47:58.554123 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.554085 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-dbxqd"] Jun 03 01:47:58.554647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.554550 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a855e1-0a2d-4658-a703-3d0a6d8404e5" containerName="registry-server" Jun 03 01:47:58.554647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.554566 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a855e1-0a2d-4658-a703-3d0a6d8404e5" containerName="registry-server" Jun 03 01:47:58.554647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.554643 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="26a855e1-0a2d-4658-a703-3d0a6d8404e5" containerName="registry-server" Jun 03 01:47:58.572048 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.572013 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:47:58.575766 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.575739 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hxdwq\"" Jun 03 01:47:58.576342 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.576317 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-dbxqd"] Jun 03 01:47:58.744404 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.744364 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxclq\" (UniqueName: \"kubernetes.io/projected/99974027-058c-4f5e-9bd4-5645db4ade12-kube-api-access-bxclq\") pod \"authorino-operator-657f44b778-dbxqd\" (UID: \"99974027-058c-4f5e-9bd4-5645db4ade12\") " pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:47:58.845320 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.845285 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxclq\" (UniqueName: \"kubernetes.io/projected/99974027-058c-4f5e-9bd4-5645db4ade12-kube-api-access-bxclq\") pod \"authorino-operator-657f44b778-dbxqd\" (UID: \"99974027-058c-4f5e-9bd4-5645db4ade12\") " pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:47:58.857429 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.857392 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxclq\" (UniqueName: \"kubernetes.io/projected/99974027-058c-4f5e-9bd4-5645db4ade12-kube-api-access-bxclq\") pod \"authorino-operator-657f44b778-dbxqd\" (UID: \"99974027-058c-4f5e-9bd4-5645db4ade12\") " pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:47:58.887221 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:58.887185 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:47:59.022954 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:59.022924 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-dbxqd"] Jun 03 01:47:59.025786 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:47:59.025753 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99974027_058c_4f5e_9bd4_5645db4ade12.slice/crio-e793f10158bcda135a2a2fd734d49f423f48657bcfd4e4dfbd1e9b690285ded9 WatchSource:0}: Error finding container e793f10158bcda135a2a2fd734d49f423f48657bcfd4e4dfbd1e9b690285ded9: Status 404 returned error can't find the container with id e793f10158bcda135a2a2fd734d49f423f48657bcfd4e4dfbd1e9b690285ded9 Jun 03 01:47:59.567123 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:47:59.567083 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" event={"ID":"99974027-058c-4f5e-9bd4-5645db4ade12","Type":"ContainerStarted","Data":"e793f10158bcda135a2a2fd734d49f423f48657bcfd4e4dfbd1e9b690285ded9"} Jun 03 01:48:01.576749 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.576641 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" event={"ID":"99974027-058c-4f5e-9bd4-5645db4ade12","Type":"ContainerStarted","Data":"8d8c0ff767dda633f52b4a21a5f5c4447a5b46b54e0f2dc8d5040b054cdbcc14"} Jun 03 01:48:01.577136 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.576811 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:48:01.595171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.595102 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" podStartSLOduration=1.437964342 podStartE2EDuration="3.59508467s" podCreationTimestamp="2026-06-03 01:47:58 +0000 UTC" firstStartedPulling="2026-06-03 01:47:59.027699379 +0000 UTC m=+449.682342638" lastFinishedPulling="2026-06-03 01:48:01.184819704 +0000 UTC m=+451.839462966" observedRunningTime="2026-06-03 01:48:01.593685788 +0000 UTC m=+452.248329097" watchObservedRunningTime="2026-06-03 01:48:01.59508467 +0000 UTC m=+452.249727952" Jun 03 01:48:01.777106 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.777075 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf"] Jun 03 01:48:01.780643 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.780624 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:01.784029 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.784004 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wf4sw\"" Jun 03 01:48:01.790030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.790000 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf"] Jun 03 01:48:01.974518 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:01.974485 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsn5g\" (UniqueName: \"kubernetes.io/projected/195565fb-a459-4b60-a15a-35118822f516-kube-api-access-zsn5g\") pod \"limitador-operator-controller-manager-85c4996f8c-2l2kf\" (UID: \"195565fb-a459-4b60-a15a-35118822f516\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:02.075831 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:02.075794 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zsn5g\" (UniqueName: \"kubernetes.io/projected/195565fb-a459-4b60-a15a-35118822f516-kube-api-access-zsn5g\") pod \"limitador-operator-controller-manager-85c4996f8c-2l2kf\" (UID: \"195565fb-a459-4b60-a15a-35118822f516\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:02.084624 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:02.084584 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsn5g\" (UniqueName: \"kubernetes.io/projected/195565fb-a459-4b60-a15a-35118822f516-kube-api-access-zsn5g\") pod \"limitador-operator-controller-manager-85c4996f8c-2l2kf\" (UID: \"195565fb-a459-4b60-a15a-35118822f516\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:02.091339 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:02.091299 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:02.227668 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:02.227584 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf"] Jun 03 01:48:02.230607 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:48:02.230561 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195565fb_a459_4b60_a15a_35118822f516.slice/crio-dd06c43ad8b9696feb293c3edf587e97ef6b2f40a94a4d661becdbbb18a0880c WatchSource:0}: Error finding container dd06c43ad8b9696feb293c3edf587e97ef6b2f40a94a4d661becdbbb18a0880c: Status 404 returned error can't find the container with id dd06c43ad8b9696feb293c3edf587e97ef6b2f40a94a4d661becdbbb18a0880c Jun 03 01:48:02.581240 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:02.581141 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" event={"ID":"195565fb-a459-4b60-a15a-35118822f516","Type":"ContainerStarted","Data":"dd06c43ad8b9696feb293c3edf587e97ef6b2f40a94a4d661becdbbb18a0880c"} Jun 03 01:48:04.591301 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:04.591255 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" event={"ID":"195565fb-a459-4b60-a15a-35118822f516","Type":"ContainerStarted","Data":"9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f"} Jun 03 01:48:04.591951 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:04.591359 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:04.611572 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:04.611485 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" podStartSLOduration=1.52684742 podStartE2EDuration="3.61146507s" podCreationTimestamp="2026-06-03 01:48:01 +0000 UTC" firstStartedPulling="2026-06-03 01:48:02.232738393 +0000 UTC m=+452.887381654" lastFinishedPulling="2026-06-03 01:48:04.317356044 +0000 UTC m=+454.971999304" observedRunningTime="2026-06-03 01:48:04.610472136 +0000 UTC m=+455.265115417" watchObservedRunningTime="2026-06-03 01:48:04.61146507 +0000 UTC m=+455.266108351" Jun 03 01:48:09.267038 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.267001 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5"] Jun 03 01:48:09.271003 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.270977 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:09.273808 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.273781 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 03 01:48:09.273808 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.273801 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sff59\"" Jun 03 01:48:09.280855 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.280828 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5"] Jun 03 01:48:09.337715 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.337679 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgl2\" (UniqueName: \"kubernetes.io/projected/858274fb-584c-4687-a3e3-e3d16130df30-kube-api-access-6qgl2\") pod \"dns-operator-controller-manager-648d5c98bc-2dqp5\" (UID: \"858274fb-584c-4687-a3e3-e3d16130df30\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:09.438744 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.438709 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgl2\" (UniqueName: \"kubernetes.io/projected/858274fb-584c-4687-a3e3-e3d16130df30-kube-api-access-6qgl2\") pod \"dns-operator-controller-manager-648d5c98bc-2dqp5\" (UID: \"858274fb-584c-4687-a3e3-e3d16130df30\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:09.447221 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.447186 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgl2\" (UniqueName: \"kubernetes.io/projected/858274fb-584c-4687-a3e3-e3d16130df30-kube-api-access-6qgl2\") pod \"dns-operator-controller-manager-648d5c98bc-2dqp5\" (UID: \"858274fb-584c-4687-a3e3-e3d16130df30\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:09.583037 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.582949 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:09.721031 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:09.721001 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5"] Jun 03 01:48:09.723490 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:48:09.723463 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858274fb_584c_4687_a3e3_e3d16130df30.slice/crio-fe67e9cbdcdd6cee68a8b72068cbe60c2d2d47b320303a1522ae7895c3e5cb79 WatchSource:0}: Error finding container fe67e9cbdcdd6cee68a8b72068cbe60c2d2d47b320303a1522ae7895c3e5cb79: Status 404 returned error can't find the container with id fe67e9cbdcdd6cee68a8b72068cbe60c2d2d47b320303a1522ae7895c3e5cb79 Jun 03 01:48:10.615805 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:10.615728 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" event={"ID":"858274fb-584c-4687-a3e3-e3d16130df30","Type":"ContainerStarted","Data":"fe67e9cbdcdd6cee68a8b72068cbe60c2d2d47b320303a1522ae7895c3e5cb79"} Jun 03 01:48:12.584201 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:12.584113 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-dbxqd" Jun 03 01:48:12.625355 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:12.625307 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" event={"ID":"858274fb-584c-4687-a3e3-e3d16130df30","Type":"ContainerStarted","Data":"594a9d03af4f12f4c8eaf18a458274ee8b8c1e3ae99b5e834b1aed889ae61080"} Jun 03 01:48:12.625577 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:12.625486 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:12.647795 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:12.647732 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" podStartSLOduration=1.095884593 podStartE2EDuration="3.647715085s" podCreationTimestamp="2026-06-03 01:48:09 +0000 UTC" firstStartedPulling="2026-06-03 01:48:09.725309754 +0000 UTC m=+460.379953012" lastFinishedPulling="2026-06-03 01:48:12.277140246 +0000 UTC m=+462.931783504" observedRunningTime="2026-06-03 01:48:12.646177208 +0000 UTC m=+463.300820489" watchObservedRunningTime="2026-06-03 01:48:12.647715085 +0000 UTC m=+463.302358364" Jun 03 01:48:15.597583 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:15.597545 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:23.631597 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:23.631560 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-2dqp5" Jun 03 01:48:25.381189 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.381151 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp"] Jun 03 01:48:25.384905 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.384880 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.387551 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.387513 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vsx6t\"" Jun 03 01:48:25.395443 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.395413 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp"] Jun 03 01:48:25.484961 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.484915 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkrn6\" (UniqueName: \"kubernetes.io/projected/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-kube-api-access-wkrn6\") pod \"kuadrant-operator-controller-manager-84b657d985-krfvp\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.485141 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.485037 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-krfvp\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.586305 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.586257 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wkrn6\" (UniqueName: \"kubernetes.io/projected/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-kube-api-access-wkrn6\") pod \"kuadrant-operator-controller-manager-84b657d985-krfvp\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.586490 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.586352 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-krfvp\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.586752 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.586734 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-krfvp\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.598840 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.598806 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkrn6\" (UniqueName: \"kubernetes.io/projected/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-kube-api-access-wkrn6\") pod \"kuadrant-operator-controller-manager-84b657d985-krfvp\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.680074 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.679981 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp"] Jun 03 01:48:25.680356 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.680331 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:25.700586 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.700547 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp"] Jun 03 01:48:25.708938 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.708891 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7"] Jun 03 01:48:25.714172 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.714129 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf"] Jun 03 01:48:25.714355 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.714278 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.714461 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.714431 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" podUID="195565fb-a459-4b60-a15a-35118822f516" containerName="manager" containerID="cri-o://9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f" gracePeriod=2 Jun 03 01:48:25.724390 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.723991 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf"] Jun 03 01:48:25.726789 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.726211 2553 status_manager.go:895] "Failed to get status for pod" podUID="195565fb-a459-4b60-a15a-35118822f516" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" err="pods \"limitador-operator-controller-manager-85c4996f8c-2l2kf\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:25.729613 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.729582 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7"] Jun 03 01:48:25.739604 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.739570 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78"] Jun 03 01:48:25.740046 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.740028 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195565fb-a459-4b60-a15a-35118822f516" containerName="manager" Jun 03 01:48:25.740130 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.740049 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="195565fb-a459-4b60-a15a-35118822f516" containerName="manager" Jun 03 01:48:25.740183 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.740133 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="195565fb-a459-4b60-a15a-35118822f516" containerName="manager" Jun 03 01:48:25.743132 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.743111 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:25.750845 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.750811 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78"] Jun 03 01:48:25.759209 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.759166 2553 status_manager.go:895] "Failed to get status for pod" podUID="195565fb-a459-4b60-a15a-35118822f516" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" err="pods \"limitador-operator-controller-manager-85c4996f8c-2l2kf\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:25.788259 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.788218 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsx5b\" (UniqueName: \"kubernetes.io/projected/c6d9f54b-bb75-40dd-8843-128bddded52e-kube-api-access-rsx5b\") pod \"limitador-operator-controller-manager-85c4996f8c-b7d78\" (UID: \"c6d9f54b-bb75-40dd-8843-128bddded52e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:25.788408 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.788352 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc53c037-0eb7-449c-8d33-cd447b9982fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-4pvh7\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.788475 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.788425 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4pws\" (UniqueName: \"kubernetes.io/projected/dc53c037-0eb7-449c-8d33-cd447b9982fc-kube-api-access-r4pws\") pod \"kuadrant-operator-controller-manager-84b657d985-4pvh7\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.889993 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.889943 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc53c037-0eb7-449c-8d33-cd447b9982fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-4pvh7\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.890137 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.890041 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r4pws\" (UniqueName: \"kubernetes.io/projected/dc53c037-0eb7-449c-8d33-cd447b9982fc-kube-api-access-r4pws\") pod \"kuadrant-operator-controller-manager-84b657d985-4pvh7\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.890137 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.890077 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rsx5b\" (UniqueName: \"kubernetes.io/projected/c6d9f54b-bb75-40dd-8843-128bddded52e-kube-api-access-rsx5b\") pod \"limitador-operator-controller-manager-85c4996f8c-b7d78\" (UID: \"c6d9f54b-bb75-40dd-8843-128bddded52e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:25.890343 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.890323 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc53c037-0eb7-449c-8d33-cd447b9982fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-4pvh7\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.899917 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.899878 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsx5b\" (UniqueName: \"kubernetes.io/projected/c6d9f54b-bb75-40dd-8843-128bddded52e-kube-api-access-rsx5b\") pod \"limitador-operator-controller-manager-85c4996f8c-b7d78\" (UID: \"c6d9f54b-bb75-40dd-8843-128bddded52e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:25.900321 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.900292 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4pws\" (UniqueName: \"kubernetes.io/projected/dc53c037-0eb7-449c-8d33-cd447b9982fc-kube-api-access-r4pws\") pod \"kuadrant-operator-controller-manager-84b657d985-4pvh7\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:25.949427 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.949401 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:25.991277 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.991242 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsn5g\" (UniqueName: \"kubernetes.io/projected/195565fb-a459-4b60-a15a-35118822f516-kube-api-access-zsn5g\") pod \"195565fb-a459-4b60-a15a-35118822f516\" (UID: \"195565fb-a459-4b60-a15a-35118822f516\") " Jun 03 01:48:25.993449 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:25.993414 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195565fb-a459-4b60-a15a-35118822f516-kube-api-access-zsn5g" (OuterVolumeSpecName: "kube-api-access-zsn5g") pod "195565fb-a459-4b60-a15a-35118822f516" (UID: "195565fb-a459-4b60-a15a-35118822f516"). InnerVolumeSpecName "kube-api-access-zsn5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:48:26.092944 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.092900 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsn5g\" (UniqueName: \"kubernetes.io/projected/195565fb-a459-4b60-a15a-35118822f516-kube-api-access-zsn5g\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:48:26.104792 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.104755 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:26.110658 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.110631 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:26.266484 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.266444 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7"] Jun 03 01:48:26.270436 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:48:26.270409 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc53c037_0eb7_449c_8d33_cd447b9982fc.slice/crio-defd274fc7cb63b9a2a986a7c6b162893f2f94cd6bf47909ca5698f98b79547c WatchSource:0}: Error finding container defd274fc7cb63b9a2a986a7c6b162893f2f94cd6bf47909ca5698f98b79547c: Status 404 returned error can't find the container with id defd274fc7cb63b9a2a986a7c6b162893f2f94cd6bf47909ca5698f98b79547c Jun 03 01:48:26.293494 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.293461 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78"] Jun 03 01:48:26.295366 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:48:26.295338 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d9f54b_bb75_40dd_8843_128bddded52e.slice/crio-314dbe0b8aee77491a1897e2a302a1461ea9c7aa95bb595b971402b0e2362f50 WatchSource:0}: Error finding container 314dbe0b8aee77491a1897e2a302a1461ea9c7aa95bb595b971402b0e2362f50: Status 404 returned error can't find the container with id 314dbe0b8aee77491a1897e2a302a1461ea9c7aa95bb595b971402b0e2362f50 Jun 03 01:48:26.677297 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.677259 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" event={"ID":"dc53c037-0eb7-449c-8d33-cd447b9982fc","Type":"ContainerStarted","Data":"defd274fc7cb63b9a2a986a7c6b162893f2f94cd6bf47909ca5698f98b79547c"} Jun 03 01:48:26.678824 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.678786 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" event={"ID":"c6d9f54b-bb75-40dd-8843-128bddded52e","Type":"ContainerStarted","Data":"4fd765094a81b3f11147dd5811862939763d3b642d1cbcb4793627baeb2ca1f8"} Jun 03 01:48:26.678824 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.678823 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" event={"ID":"c6d9f54b-bb75-40dd-8843-128bddded52e","Type":"ContainerStarted","Data":"314dbe0b8aee77491a1897e2a302a1461ea9c7aa95bb595b971402b0e2362f50"} Jun 03 01:48:26.679050 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.678877 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:26.679870 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.679846 2553 generic.go:358] "Generic (PLEG): container finished" podID="195565fb-a459-4b60-a15a-35118822f516" containerID="9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f" exitCode=0 Jun 03 01:48:26.679942 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.679887 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2l2kf" Jun 03 01:48:26.679975 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.679944 2553 scope.go:117] "RemoveContainer" containerID="9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f" Jun 03 01:48:26.689666 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.689386 2553 scope.go:117] "RemoveContainer" containerID="9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f" Jun 03 01:48:26.689749 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:48:26.689717 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f\": container with ID starting with 9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f not found: ID does not exist" containerID="9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f" Jun 03 01:48:26.689749 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.689740 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f"} err="failed to get container status \"9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f\": rpc error: code = NotFound desc = could not find container \"9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f\": container with ID starting with 9c0247d16014df3f21795f355b9a503884580d53beb67981dabba19cf389fe7f not found: ID does not exist" Jun 03 01:48:26.697131 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:26.697062 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" podStartSLOduration=1.6970457049999998 podStartE2EDuration="1.697045705s" podCreationTimestamp="2026-06-03 01:48:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:48:26.695081004 +0000 UTC m=+477.349724295" watchObservedRunningTime="2026-06-03 01:48:26.697045705 +0000 UTC m=+477.351688985" Jun 03 01:48:27.942687 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:27.941221 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="195565fb-a459-4b60-a15a-35118822f516" path="/var/lib/kubelet/pods/195565fb-a459-4b60-a15a-35118822f516/volumes" Jun 03 01:48:28.213479 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:48:28.213427 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bfcedc_96e7_46cf_abae_15dc6fe6a3c4.slice/crio-4644a830e9b8e0bf858973a3d640be08abf16776861a7ec855f1bd575a8333a1 WatchSource:0}: Error finding container 4644a830e9b8e0bf858973a3d640be08abf16776861a7ec855f1bd575a8333a1: Status 404 returned error can't find the container with id 4644a830e9b8e0bf858973a3d640be08abf16776861a7ec855f1bd575a8333a1 Jun 03 01:48:31.706342 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.706305 2553 generic.go:358] "Generic (PLEG): container finished" podID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" containerID="8554475c88ef82c020d596e962c79caf9bccada1632c6bf75b302d1db5586094" exitCode=1 Jun 03 01:48:31.708119 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.708093 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" event={"ID":"dc53c037-0eb7-449c-8d33-cd447b9982fc","Type":"ContainerStarted","Data":"0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684"} Jun 03 01:48:31.708245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.708234 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:31.708916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.708884 2553 status_manager.go:895] "Failed to get status for pod" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" err="pods \"kuadrant-operator-controller-manager-84b657d985-krfvp\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:31.711032 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.710995 2553 status_manager.go:895] "Failed to get status for pod" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" err="pods \"kuadrant-operator-controller-manager-84b657d985-krfvp\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:31.730464 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.730414 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" podStartSLOduration=2.189339013 podStartE2EDuration="6.730397094s" podCreationTimestamp="2026-06-03 01:48:25 +0000 UTC" firstStartedPulling="2026-06-03 01:48:26.272687671 +0000 UTC m=+476.927330929" lastFinishedPulling="2026-06-03 01:48:30.813745749 +0000 UTC m=+481.468389010" observedRunningTime="2026-06-03 01:48:31.728733626 +0000 UTC m=+482.383376906" watchObservedRunningTime="2026-06-03 01:48:31.730397094 +0000 UTC m=+482.385040353" Jun 03 01:48:31.743036 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.743009 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:31.745313 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.745282 2553 status_manager.go:895] "Failed to get status for pod" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" err="pods \"kuadrant-operator-controller-manager-84b657d985-krfvp\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:31.851522 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.851489 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkrn6\" (UniqueName: \"kubernetes.io/projected/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-kube-api-access-wkrn6\") pod \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " Jun 03 01:48:31.851708 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.851620 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-extensions-socket-volume\") pod \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\" (UID: \"f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4\") " Jun 03 01:48:31.851933 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.851908 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" (UID: "f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 01:48:31.853891 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.853865 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-kube-api-access-wkrn6" (OuterVolumeSpecName: "kube-api-access-wkrn6") pod "f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" (UID: "f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4"). InnerVolumeSpecName "kube-api-access-wkrn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:48:31.939852 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.939816 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" path="/var/lib/kubelet/pods/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4/volumes" Jun 03 01:48:31.953213 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.953182 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkrn6\" (UniqueName: \"kubernetes.io/projected/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-kube-api-access-wkrn6\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:48:31.953213 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:31.953213 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4-extensions-socket-volume\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:48:32.715382 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:32.715351 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" Jun 03 01:48:32.715849 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:32.715379 2553 scope.go:117] "RemoveContainer" containerID="8554475c88ef82c020d596e962c79caf9bccada1632c6bf75b302d1db5586094" Jun 03 01:48:32.718225 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:32.718191 2553 status_manager.go:895] "Failed to get status for pod" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" err="pods \"kuadrant-operator-controller-manager-84b657d985-krfvp\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:32.720478 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:32.720445 2553 status_manager.go:895] "Failed to get status for pod" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-krfvp" err="pods \"kuadrant-operator-controller-manager-84b657d985-krfvp\" is forbidden: User \"system:node:ip-10-0-136-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-201.ec2.internal' and this object" Jun 03 01:48:37.689598 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:37.689563 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-b7d78" Jun 03 01:48:42.717714 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:42.717679 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:55.072846 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.072804 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7"] Jun 03 01:48:55.073260 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.073041 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" podUID="dc53c037-0eb7-449c-8d33-cd447b9982fc" containerName="manager" containerID="cri-o://0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684" gracePeriod=10 Jun 03 01:48:55.332670 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.332599 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:55.369543 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.369494 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4pws\" (UniqueName: \"kubernetes.io/projected/dc53c037-0eb7-449c-8d33-cd447b9982fc-kube-api-access-r4pws\") pod \"dc53c037-0eb7-449c-8d33-cd447b9982fc\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " Jun 03 01:48:55.369708 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.369627 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc53c037-0eb7-449c-8d33-cd447b9982fc-extensions-socket-volume\") pod \"dc53c037-0eb7-449c-8d33-cd447b9982fc\" (UID: \"dc53c037-0eb7-449c-8d33-cd447b9982fc\") " Jun 03 01:48:55.370134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.370101 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc53c037-0eb7-449c-8d33-cd447b9982fc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc53c037-0eb7-449c-8d33-cd447b9982fc" (UID: "dc53c037-0eb7-449c-8d33-cd447b9982fc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 01:48:55.371748 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.371726 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc53c037-0eb7-449c-8d33-cd447b9982fc-kube-api-access-r4pws" (OuterVolumeSpecName: "kube-api-access-r4pws") pod "dc53c037-0eb7-449c-8d33-cd447b9982fc" (UID: "dc53c037-0eb7-449c-8d33-cd447b9982fc"). InnerVolumeSpecName "kube-api-access-r4pws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:48:55.470602 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.470559 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc53c037-0eb7-449c-8d33-cd447b9982fc-extensions-socket-volume\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:48:55.470602 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.470597 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4pws\" (UniqueName: \"kubernetes.io/projected/dc53c037-0eb7-449c-8d33-cd447b9982fc-kube-api-access-r4pws\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:48:55.802893 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.802856 2553 generic.go:358] "Generic (PLEG): container finished" podID="dc53c037-0eb7-449c-8d33-cd447b9982fc" containerID="0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684" exitCode=0 Jun 03 01:48:55.803091 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.802927 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" Jun 03 01:48:55.803091 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.802938 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" event={"ID":"dc53c037-0eb7-449c-8d33-cd447b9982fc","Type":"ContainerDied","Data":"0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684"} Jun 03 01:48:55.803091 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.802977 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7" event={"ID":"dc53c037-0eb7-449c-8d33-cd447b9982fc","Type":"ContainerDied","Data":"defd274fc7cb63b9a2a986a7c6b162893f2f94cd6bf47909ca5698f98b79547c"} Jun 03 01:48:55.803091 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.802992 2553 scope.go:117] "RemoveContainer" containerID="0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684" Jun 03 01:48:55.812411 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.812391 2553 scope.go:117] "RemoveContainer" containerID="0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684" Jun 03 01:48:55.812718 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:48:55.812701 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684\": container with ID starting with 0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684 not found: ID does not exist" containerID="0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684" Jun 03 01:48:55.812772 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.812729 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684"} err="failed to get container status \"0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684\": rpc error: code = NotFound desc = could not find container \"0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684\": container with ID starting with 0cafd21d35b71541cb610bad1bbebd9f9f8043dc047f8326ec8f4b6c8fdb0684 not found: ID does not exist" Jun 03 01:48:55.827642 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.827609 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7"] Jun 03 01:48:55.830999 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.830973 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-4pvh7"] Jun 03 01:48:55.940018 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:55.939985 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc53c037-0eb7-449c-8d33-cd447b9982fc" path="/var/lib/kubelet/pods/dc53c037-0eb7-449c-8d33-cd447b9982fc/volumes" Jun 03 01:48:59.113131 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113098 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f"] Jun 03 01:48:59.113720 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113698 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" containerName="manager" Jun 03 01:48:59.113809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113721 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" containerName="manager" Jun 03 01:48:59.113809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113772 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc53c037-0eb7-449c-8d33-cd447b9982fc" containerName="manager" Jun 03 01:48:59.113809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113781 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc53c037-0eb7-449c-8d33-cd447b9982fc" containerName="manager" Jun 03 01:48:59.113954 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113877 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="f2bfcedc-96e7-46cf-abae-15dc6fe6a3c4" containerName="manager" Jun 03 01:48:59.113954 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.113895 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="dc53c037-0eb7-449c-8d33-cd447b9982fc" containerName="manager" Jun 03 01:48:59.119504 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.119471 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.122394 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.122362 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-l577s\"" Jun 03 01:48:59.128788 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.128761 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f"] Jun 03 01:48:59.204936 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.204890 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205115 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.204961 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205115 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.204982 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205115 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.205022 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205115 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.205040 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205331 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.205106 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205331 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.205206 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2dms\" (UniqueName: \"kubernetes.io/projected/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-kube-api-access-k2dms\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205331 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.205270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.205331 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.205297 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.306757 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.306714 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2dms\" (UniqueName: \"kubernetes.io/projected/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-kube-api-access-k2dms\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.306947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.306783 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.306947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.306811 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.306947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.306844 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.306947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.306915 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.306947 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.306942 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307152 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307111 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307217 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307169 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307268 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307247 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307326 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307277 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307379 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307335 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307434 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307418 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307632 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307605 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.307809 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.307789 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.309478 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.309445 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.309478 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.309473 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.315044 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.315022 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.315216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.315197 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2dms\" (UniqueName: \"kubernetes.io/projected/2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0-kube-api-access-k2dms\") pod \"maas-default-gateway-openshift-default-7df95f575-nlx4f\" (UID: \"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.433704 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.433607 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:48:59.572710 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.572685 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f"] Jun 03 01:48:59.574797 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:48:59.574760 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f586b8f_dd7d_4ed3_abb2_d3d758b73bb0.slice/crio-03f07c87da17a273dffb61910b0d723ee425a0b80ab2ec6ae7df62aabd010e1f WatchSource:0}: Error finding container 03f07c87da17a273dffb61910b0d723ee425a0b80ab2ec6ae7df62aabd010e1f: Status 404 returned error can't find the container with id 03f07c87da17a273dffb61910b0d723ee425a0b80ab2ec6ae7df62aabd010e1f Jun 03 01:48:59.576823 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.576793 2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 03 01:48:59.576903 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.576857 2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 03 01:48:59.576903 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.576885 2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 03 01:48:59.819516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.819423 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" event={"ID":"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0","Type":"ContainerStarted","Data":"d2000c5b15559a4849434c3e05be4c154608c0c11c00a7cbaabaf8bc92f379a2"} Jun 03 01:48:59.819516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.819466 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" event={"ID":"2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0","Type":"ContainerStarted","Data":"03f07c87da17a273dffb61910b0d723ee425a0b80ab2ec6ae7df62aabd010e1f"} Jun 03 01:48:59.840475 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:48:59.840416 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" podStartSLOduration=0.840397557 podStartE2EDuration="840.397557ms" podCreationTimestamp="2026-06-03 01:48:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 01:48:59.837802195 +0000 UTC m=+510.492445476" watchObservedRunningTime="2026-06-03 01:48:59.840397557 +0000 UTC m=+510.495040841" Jun 03 01:49:00.433957 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:00.433913 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:49:00.439990 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:00.439958 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:49:00.823629 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:00.823522 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:49:00.824647 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:00.824631 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-nlx4f" Jun 03 01:49:54.149694 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.149656 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-c65fb"] Jun 03 01:49:54.152223 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.152193 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" Jun 03 01:49:54.154704 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.154670 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 03 01:49:54.154833 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.154710 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-wrjtt\"" Jun 03 01:49:54.155775 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.155756 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 03 01:49:54.160051 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.159748 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-c65fb"] Jun 03 01:49:54.202575 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.202524 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67qsf\" (UniqueName: \"kubernetes.io/projected/d5867c1e-a601-49f1-8b3b-467e38109f26-kube-api-access-67qsf\") pod \"keycloak-operator-7864d68cbd-c65fb\" (UID: \"d5867c1e-a601-49f1-8b3b-467e38109f26\") " pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" Jun 03 01:49:54.303174 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.303141 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-67qsf\" (UniqueName: \"kubernetes.io/projected/d5867c1e-a601-49f1-8b3b-467e38109f26-kube-api-access-67qsf\") pod \"keycloak-operator-7864d68cbd-c65fb\" (UID: \"d5867c1e-a601-49f1-8b3b-467e38109f26\") " pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" Jun 03 01:49:54.315797 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.315767 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67qsf\" (UniqueName: \"kubernetes.io/projected/d5867c1e-a601-49f1-8b3b-467e38109f26-kube-api-access-67qsf\") pod \"keycloak-operator-7864d68cbd-c65fb\" (UID: \"d5867c1e-a601-49f1-8b3b-467e38109f26\") " pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" Jun 03 01:49:54.463730 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.463640 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" Jun 03 01:49:54.622466 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:54.622437 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-c65fb"] Jun 03 01:49:54.625328 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:49:54.625298 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5867c1e_a601_49f1_8b3b_467e38109f26.slice/crio-717ee5cfd3b2f9df320382f491c80b763d49bde5ff24ec3e04e61f64d12483fc WatchSource:0}: Error finding container 717ee5cfd3b2f9df320382f491c80b763d49bde5ff24ec3e04e61f64d12483fc: Status 404 returned error can't find the container with id 717ee5cfd3b2f9df320382f491c80b763d49bde5ff24ec3e04e61f64d12483fc Jun 03 01:49:55.014826 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:49:55.014789 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" event={"ID":"d5867c1e-a601-49f1-8b3b-467e38109f26","Type":"ContainerStarted","Data":"717ee5cfd3b2f9df320382f491c80b763d49bde5ff24ec3e04e61f64d12483fc"} Jun 03 01:50:01.040512 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:01.040475 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" event={"ID":"d5867c1e-a601-49f1-8b3b-467e38109f26","Type":"ContainerStarted","Data":"d1e6d3c9e27fc8c6c9a3722774f404aa47087d81c6b1ad681d5b3a49e386665a"} Jun 03 01:50:01.055720 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:01.055665 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-7864d68cbd-c65fb" podStartSLOduration=1.374748288 podStartE2EDuration="7.055649541s" podCreationTimestamp="2026-06-03 01:49:54 +0000 UTC" firstStartedPulling="2026-06-03 01:49:54.626698963 +0000 UTC m=+565.281342221" lastFinishedPulling="2026-06-03 01:50:00.307600212 +0000 UTC m=+570.962243474" observedRunningTime="2026-06-03 01:50:01.055421887 +0000 UTC m=+571.710065168" watchObservedRunningTime="2026-06-03 01:50:01.055649541 +0000 UTC m=+571.710292821" Jun 03 01:50:29.818822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:29.818790 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:50:29.821596 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:29.821576 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:50:45.505099 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.505011 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69665ccb6d-9lj4w"] Jun 03 01:50:45.507394 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.507376 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:50:45.509974 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.509947 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 03 01:50:45.510113 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.509947 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4247v\"" Jun 03 01:50:45.518095 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.518059 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69665ccb6d-9lj4w"] Jun 03 01:50:45.584504 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.584468 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggxc\" (UniqueName: \"kubernetes.io/projected/6c6c51aa-1d60-4230-be2e-3d137358d9cc-kube-api-access-6ggxc\") pod \"maas-controller-69665ccb6d-9lj4w\" (UID: \"6c6c51aa-1d60-4230-be2e-3d137358d9cc\") " pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:50:45.686061 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.686019 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggxc\" (UniqueName: \"kubernetes.io/projected/6c6c51aa-1d60-4230-be2e-3d137358d9cc-kube-api-access-6ggxc\") pod \"maas-controller-69665ccb6d-9lj4w\" (UID: \"6c6c51aa-1d60-4230-be2e-3d137358d9cc\") " pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:50:45.695061 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.695026 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggxc\" (UniqueName: \"kubernetes.io/projected/6c6c51aa-1d60-4230-be2e-3d137358d9cc-kube-api-access-6ggxc\") pod \"maas-controller-69665ccb6d-9lj4w\" (UID: \"6c6c51aa-1d60-4230-be2e-3d137358d9cc\") " pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:50:45.819614 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.819498 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:50:45.961971 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:45.961942 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69665ccb6d-9lj4w"] Jun 03 01:50:45.964357 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:50:45.964319 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6c51aa_1d60_4230_be2e_3d137358d9cc.slice/crio-cb70e69af8cf9f018dafc1309f4ee9863ce1f7ac324db76170153afbaac07662 WatchSource:0}: Error finding container cb70e69af8cf9f018dafc1309f4ee9863ce1f7ac324db76170153afbaac07662: Status 404 returned error can't find the container with id cb70e69af8cf9f018dafc1309f4ee9863ce1f7ac324db76170153afbaac07662 Jun 03 01:50:46.214601 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:46.214558 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" event={"ID":"6c6c51aa-1d60-4230-be2e-3d137358d9cc","Type":"ContainerStarted","Data":"cb70e69af8cf9f018dafc1309f4ee9863ce1f7ac324db76170153afbaac07662"} Jun 03 01:50:49.230737 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:49.230695 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" event={"ID":"6c6c51aa-1d60-4230-be2e-3d137358d9cc","Type":"ContainerStarted","Data":"8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9"} Jun 03 01:50:49.231131 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:49.230772 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:50:49.249035 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:50:49.248975 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" podStartSLOduration=2.033274142 podStartE2EDuration="4.24895689s" podCreationTimestamp="2026-06-03 01:50:45 +0000 UTC" firstStartedPulling="2026-06-03 01:50:45.96571336 +0000 UTC m=+616.620356618" lastFinishedPulling="2026-06-03 01:50:48.181396086 +0000 UTC m=+618.836039366" observedRunningTime="2026-06-03 01:50:49.247222251 +0000 UTC m=+619.901865533" watchObservedRunningTime="2026-06-03 01:50:49.24895689 +0000 UTC m=+619.903600183" Jun 03 01:51:00.241465 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:51:00.241426 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:52:26.756437 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.756335 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv"] Jun 03 01:52:26.759962 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.759942 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.762727 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.762696 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-msbjn\"" Jun 03 01:52:26.763801 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.763781 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 03 01:52:26.763924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.763858 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 03 01:52:26.763924 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.763866 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 03 01:52:26.769314 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.769280 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv"] Jun 03 01:52:26.837171 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.837136 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqpx2\" (UniqueName: \"kubernetes.io/projected/1791040f-f631-4df1-b6e7-d12d81b2ed78-kube-api-access-bqpx2\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.837368 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.837198 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.837368 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.837238 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.837368 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.837283 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.837368 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.837305 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1791040f-f631-4df1-b6e7-d12d81b2ed78-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.837520 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.837394 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.938568 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.938514 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.938765 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.938586 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bqpx2\" (UniqueName: \"kubernetes.io/projected/1791040f-f631-4df1-b6e7-d12d81b2ed78-kube-api-access-bqpx2\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.938765 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.938629 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.938765 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.938656 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.938765 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.938721 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.938765 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.938742 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1791040f-f631-4df1-b6e7-d12d81b2ed78-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.939069 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.939043 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.939134 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.939086 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.939272 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.939248 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.941127 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.941108 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1791040f-f631-4df1-b6e7-d12d81b2ed78-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.941431 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.941413 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1791040f-f631-4df1-b6e7-d12d81b2ed78-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:26.949165 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:26.949132 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqpx2\" (UniqueName: \"kubernetes.io/projected/1791040f-f631-4df1-b6e7-d12d81b2ed78-kube-api-access-bqpx2\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-44vxv\" (UID: \"1791040f-f631-4df1-b6e7-d12d81b2ed78\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:27.072822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:27.072728 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:27.210916 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:27.210888 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv"] Jun 03 01:52:27.213604 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:52:27.213570 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1791040f_f631_4df1_b6e7_d12d81b2ed78.slice/crio-75075d388f15ba38e826763483a0961516026955fa882e6a3be019fe6660e8b6 WatchSource:0}: Error finding container 75075d388f15ba38e826763483a0961516026955fa882e6a3be019fe6660e8b6: Status 404 returned error can't find the container with id 75075d388f15ba38e826763483a0961516026955fa882e6a3be019fe6660e8b6 Jun 03 01:52:27.215501 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:27.215483 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 01:52:27.600146 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:27.600109 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" event={"ID":"1791040f-f631-4df1-b6e7-d12d81b2ed78","Type":"ContainerStarted","Data":"75075d388f15ba38e826763483a0961516026955fa882e6a3be019fe6660e8b6"} Jun 03 01:52:31.619345 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:31.619309 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" event={"ID":"1791040f-f631-4df1-b6e7-d12d81b2ed78","Type":"ContainerStarted","Data":"d367c438a362600f4655861b72735cb6e4e860abbb37b924d5a5e9684a72fbf8"} Jun 03 01:52:32.625421 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:32.625324 2553 generic.go:358] "Generic (PLEG): container finished" podID="1791040f-f631-4df1-b6e7-d12d81b2ed78" containerID="d367c438a362600f4655861b72735cb6e4e860abbb37b924d5a5e9684a72fbf8" exitCode=0 Jun 03 01:52:32.625421 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:32.625362 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" event={"ID":"1791040f-f631-4df1-b6e7-d12d81b2ed78","Type":"ContainerDied","Data":"d367c438a362600f4655861b72735cb6e4e860abbb37b924d5a5e9684a72fbf8"} Jun 03 01:52:34.634860 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:34.634826 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" event={"ID":"1791040f-f631-4df1-b6e7-d12d81b2ed78","Type":"ContainerStarted","Data":"c32956b29f9c75be35abf8b74abebe231940ea118c368b613ec1083a9042dd6b"} Jun 03 01:52:34.652866 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:34.652815 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" podStartSLOduration=2.141172228 podStartE2EDuration="8.652800365s" podCreationTimestamp="2026-06-03 01:52:26 +0000 UTC" firstStartedPulling="2026-06-03 01:52:27.215639959 +0000 UTC m=+717.870283217" lastFinishedPulling="2026-06-03 01:52:33.727268082 +0000 UTC m=+724.381911354" observedRunningTime="2026-06-03 01:52:34.651082221 +0000 UTC m=+725.305725501" watchObservedRunningTime="2026-06-03 01:52:34.652800365 +0000 UTC m=+725.307443644" Jun 03 01:52:37.073632 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:37.073591 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:37.074034 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:37.073738 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:37.086419 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:37.086393 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:52:37.660202 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:52:37.660174 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-44vxv" Jun 03 01:53:06.993501 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:06.993467 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99"] Jun 03 01:53:06.998030 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:06.998008 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.000676 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.000621 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 03 01:53:07.007221 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.007194 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99"] Jun 03 01:53:07.020190 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.020159 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.020390 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.020370 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mk4b\" (UniqueName: \"kubernetes.io/projected/a47aa99d-3f15-409a-ae1a-b0ad5701586e-kube-api-access-6mk4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.020457 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.020423 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47aa99d-3f15-409a-ae1a-b0ad5701586e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.020518 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.020453 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.020655 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.020626 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.020731 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.020708 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.121516 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121478 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.121734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121526 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.121734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121611 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.121734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121655 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6mk4b\" (UniqueName: \"kubernetes.io/projected/a47aa99d-3f15-409a-ae1a-b0ad5701586e-kube-api-access-6mk4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.121734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121686 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47aa99d-3f15-409a-ae1a-b0ad5701586e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.121734 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121712 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.122010 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.121985 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.122068 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.122013 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.122119 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.122100 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.124245 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.124221 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47aa99d-3f15-409a-ae1a-b0ad5701586e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.124463 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.124448 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47aa99d-3f15-409a-ae1a-b0ad5701586e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.129956 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.129928 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mk4b\" (UniqueName: \"kubernetes.io/projected/a47aa99d-3f15-409a-ae1a-b0ad5701586e-kube-api-access-6mk4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99\" (UID: \"a47aa99d-3f15-409a-ae1a-b0ad5701586e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.310178 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.310077 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:07.454507 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.454477 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99"] Jun 03 01:53:07.458323 ip-10-0-136-201 kubenswrapper[2553]: W0603 01:53:07.456722 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47aa99d_3f15_409a_ae1a_b0ad5701586e.slice/crio-9f7d3ddcc8c8318bf2fc557e304ec5f68cd8548c60a2458037f7d63a485de6aa WatchSource:0}: Error finding container 9f7d3ddcc8c8318bf2fc557e304ec5f68cd8548c60a2458037f7d63a485de6aa: Status 404 returned error can't find the container with id 9f7d3ddcc8c8318bf2fc557e304ec5f68cd8548c60a2458037f7d63a485de6aa Jun 03 01:53:07.761202 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.761163 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" event={"ID":"a47aa99d-3f15-409a-ae1a-b0ad5701586e","Type":"ContainerStarted","Data":"74c3cda90489291501daa3ec9edc607023a7a6d7b7719d432ea49cea18db2878"} Jun 03 01:53:07.761202 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:07.761201 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" event={"ID":"a47aa99d-3f15-409a-ae1a-b0ad5701586e","Type":"ContainerStarted","Data":"9f7d3ddcc8c8318bf2fc557e304ec5f68cd8548c60a2458037f7d63a485de6aa"} Jun 03 01:53:09.772180 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:09.772138 2553 generic.go:358] "Generic (PLEG): container finished" podID="a47aa99d-3f15-409a-ae1a-b0ad5701586e" containerID="74c3cda90489291501daa3ec9edc607023a7a6d7b7719d432ea49cea18db2878" exitCode=0 Jun 03 01:53:09.772595 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:09.772210 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" event={"ID":"a47aa99d-3f15-409a-ae1a-b0ad5701586e","Type":"ContainerDied","Data":"74c3cda90489291501daa3ec9edc607023a7a6d7b7719d432ea49cea18db2878"} Jun 03 01:53:10.777863 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:10.777830 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" event={"ID":"a47aa99d-3f15-409a-ae1a-b0ad5701586e","Type":"ContainerStarted","Data":"f82331c7b9d9465af52c124bb777349830348d188a9227a8ce9828c55c54bc02"} Jun 03 01:53:10.795317 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:10.795255 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" podStartSLOduration=4.586079539 podStartE2EDuration="4.795236479s" podCreationTimestamp="2026-06-03 01:53:06 +0000 UTC" firstStartedPulling="2026-06-03 01:53:09.772833348 +0000 UTC m=+760.427476606" lastFinishedPulling="2026-06-03 01:53:09.981990287 +0000 UTC m=+760.636633546" observedRunningTime="2026-06-03 01:53:10.794062904 +0000 UTC m=+761.448706185" watchObservedRunningTime="2026-06-03 01:53:10.795236479 +0000 UTC m=+761.449879761" Jun 03 01:53:17.310692 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:17.310632 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:17.310692 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:17.310687 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:17.323744 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:17.323716 2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:53:17.815882 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:53:17.815847 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99" Jun 03 01:55:29.850054 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:29.849983 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:55:29.850054 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:29.850045 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 01:55:47.081203 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.081166 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69665ccb6d-9lj4w"] Jun 03 01:55:47.083811 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.081406 2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" podUID="6c6c51aa-1d60-4230-be2e-3d137358d9cc" containerName="manager" containerID="cri-o://8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9" gracePeriod=30 Jun 03 01:55:47.352456 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.352427 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:55:47.384435 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.384399 2553 generic.go:358] "Generic (PLEG): container finished" podID="6c6c51aa-1d60-4230-be2e-3d137358d9cc" containerID="8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9" exitCode=0 Jun 03 01:55:47.384636 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.384488 2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" Jun 03 01:55:47.384636 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.384487 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" event={"ID":"6c6c51aa-1d60-4230-be2e-3d137358d9cc","Type":"ContainerDied","Data":"8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9"} Jun 03 01:55:47.384636 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.384590 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69665ccb6d-9lj4w" event={"ID":"6c6c51aa-1d60-4230-be2e-3d137358d9cc","Type":"ContainerDied","Data":"cb70e69af8cf9f018dafc1309f4ee9863ce1f7ac324db76170153afbaac07662"} Jun 03 01:55:47.384636 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.384620 2553 scope.go:117] "RemoveContainer" containerID="8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9" Jun 03 01:55:47.393749 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.393728 2553 scope.go:117] "RemoveContainer" containerID="8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9" Jun 03 01:55:47.394073 ip-10-0-136-201 kubenswrapper[2553]: E0603 01:55:47.394040 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9\": container with ID starting with 8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9 not found: ID does not exist" containerID="8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9" Jun 03 01:55:47.394155 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.394078 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9"} err="failed to get container status \"8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9\": rpc error: code = NotFound desc = could not find container \"8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9\": container with ID starting with 8cef2fe7805fd983469690de8144f210b078d2e036f640137603310c8f73e0d9 not found: ID does not exist" Jun 03 01:55:47.482822 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.482782 2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ggxc\" (UniqueName: \"kubernetes.io/projected/6c6c51aa-1d60-4230-be2e-3d137358d9cc-kube-api-access-6ggxc\") pod \"6c6c51aa-1d60-4230-be2e-3d137358d9cc\" (UID: \"6c6c51aa-1d60-4230-be2e-3d137358d9cc\") " Jun 03 01:55:47.485216 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.485187 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c6c51aa-1d60-4230-be2e-3d137358d9cc-kube-api-access-6ggxc" (OuterVolumeSpecName: "kube-api-access-6ggxc") pod "6c6c51aa-1d60-4230-be2e-3d137358d9cc" (UID: "6c6c51aa-1d60-4230-be2e-3d137358d9cc"). InnerVolumeSpecName "kube-api-access-6ggxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 01:55:47.584310 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.584217 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ggxc\" (UniqueName: \"kubernetes.io/projected/6c6c51aa-1d60-4230-be2e-3d137358d9cc-kube-api-access-6ggxc\") on node \"ip-10-0-136-201.ec2.internal\" DevicePath \"\"" Jun 03 01:55:47.708490 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.708457 2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69665ccb6d-9lj4w"] Jun 03 01:55:47.720388 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.720356 2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69665ccb6d-9lj4w"] Jun 03 01:55:47.940678 ip-10-0-136-201 kubenswrapper[2553]: I0603 01:55:47.940636 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6c51aa-1d60-4230-be2e-3d137358d9cc" path="/var/lib/kubelet/pods/6c6c51aa-1d60-4230-be2e-3d137358d9cc/volumes" Jun 03 02:00:29.884072 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:00:29.884044 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:00:29.884711 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:00:29.884690 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:05:29.912430 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:05:29.912394 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:05:29.912990 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:05:29.912805 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:10:29.940692 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:10:29.940666 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:10:29.941571 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:10:29.941526 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:15:29.967216 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:15:29.967187 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:15:29.971001 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:15:29.970980 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:17:28.488379 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:28.488349 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7b94c665b4-n5hkv_28135a58-5e1b-49ad-aa8d-09c8296676ce/nginx/0.log" Jun 03 02:17:29.755655 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:29.755624 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf_cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7/copy-manifests/0.log" Jun 03 02:17:29.768642 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:29.768605 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf_cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7/manager/0.log" Jun 03 02:17:31.274145 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:31.274104 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-dbxqd_99974027-058c-4f5e-9bd4-5645db4ade12/manager/0.log" Jun 03 02:17:31.378437 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:31.378390 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-2dqp5_858274fb-584c-4687-a3e3-e3d16130df30/manager/0.log" Jun 03 02:17:31.599808 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:31.599772 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mmlwd_d90d35e8-e43d-4d8c-ad85-2808dfcb7135/registry-server/0.log" Jun 03 02:17:31.965796 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:31.965712 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-b7d78_c6d9f54b-bb75-40dd-8843-128bddded52e/manager/0.log" Jun 03 02:17:32.311207 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:32.311121 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f8qhjnf_18f717c7-f71f-451f-8e35-68cf623ff703/istio-proxy/0.log" Jun 03 02:17:32.772862 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:32.772831 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-nlx4f_2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0/istio-proxy/0.log" Jun 03 02:17:33.587361 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:33.587332 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-44vxv_1791040f-f631-4df1-b6e7-d12d81b2ed78/storage-initializer/0.log" Jun 03 02:17:33.595121 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:33.595098 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-44vxv_1791040f-f631-4df1-b6e7-d12d81b2ed78/main/0.log" Jun 03 02:17:33.830925 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:33.830894 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99_a47aa99d-3f15-409a-ae1a-b0ad5701586e/storage-initializer/0.log" Jun 03 02:17:33.837948 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:33.837880 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f92mw99_a47aa99d-3f15-409a-ae1a-b0ad5701586e/main/0.log" Jun 03 02:17:41.001519 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:41.001489 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-49j2x_b50f9317-b857-416c-b66a-1d7c38aa3c76/global-pull-secret-syncer/0.log" Jun 03 02:17:41.192059 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:41.192029 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pjvzl_b1e82b37-06ce-4455-924f-8c132622eae1/konnectivity-agent/0.log" Jun 03 02:17:41.239383 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:41.239348 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-201.ec2.internal_8d060eb54ad6b2dfc1cc515cf81694a9/haproxy/0.log" Jun 03 02:17:45.384996 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:45.384911 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-dbxqd_99974027-058c-4f5e-9bd4-5645db4ade12/manager/0.log" Jun 03 02:17:45.408981 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:45.408954 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-2dqp5_858274fb-584c-4687-a3e3-e3d16130df30/manager/0.log" Jun 03 02:17:45.472747 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:45.472718 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mmlwd_d90d35e8-e43d-4d8c-ad85-2808dfcb7135/registry-server/0.log" Jun 03 02:17:45.641371 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:45.641280 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-b7d78_c6d9f54b-bb75-40dd-8843-128bddded52e/manager/0.log" Jun 03 02:17:47.130118 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.130085 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-qdqbp_b5fd2a5d-fb65-4287-9a46-6444766e670b/kube-state-metrics/0.log" Jun 03 02:17:47.147353 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.147327 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-qdqbp_b5fd2a5d-fb65-4287-9a46-6444766e670b/kube-rbac-proxy-main/0.log" Jun 03 02:17:47.170698 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.170670 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-qdqbp_b5fd2a5d-fb65-4287-9a46-6444766e670b/kube-rbac-proxy-self/0.log" Jun 03 02:17:47.196849 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.196818 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5fb755fc7-zjx28_be4bb435-e6b7-4b28-86f9-4d7d059b452f/metrics-server/0.log" Jun 03 02:17:47.222804 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.222772 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-2cngq_84da1117-279d-42b4-92d2-831d29e246c8/monitoring-plugin/0.log" Jun 03 02:17:47.292481 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.292451 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-86spt_9b1aac16-1196-4e6c-a811-b5b02e2c08e1/node-exporter/0.log" Jun 03 02:17:47.315383 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.315299 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-86spt_9b1aac16-1196-4e6c-a811-b5b02e2c08e1/kube-rbac-proxy/0.log" Jun 03 02:17:47.335482 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.335455 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-86spt_9b1aac16-1196-4e6c-a811-b5b02e2c08e1/init-textfile/0.log" Jun 03 02:17:47.511819 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.511785 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-f75sv_0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e/kube-rbac-proxy-main/0.log" Jun 03 02:17:47.532033 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.531995 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-f75sv_0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e/kube-rbac-proxy-self/0.log" Jun 03 02:17:47.552112 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.552072 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-f75sv_0fe4f4ab-c16a-4b1c-bef1-4effcd5c4f6e/openshift-state-metrics/0.log" Jun 03 02:17:47.586351 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.586328 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/prometheus/0.log" Jun 03 02:17:47.607755 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.607718 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/config-reloader/0.log" Jun 03 02:17:47.631217 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.631187 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/thanos-sidecar/0.log" Jun 03 02:17:47.655263 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.654979 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/kube-rbac-proxy-web/0.log" Jun 03 02:17:47.680409 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.680378 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/kube-rbac-proxy/0.log" Jun 03 02:17:47.706262 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.706238 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/kube-rbac-proxy-thanos/0.log" Jun 03 02:17:47.727790 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.727760 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb3b9ce4-4c27-41d8-916a-cec4dcd7e385/init-config-reloader/0.log" Jun 03 02:17:47.756011 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.755962 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-dxdzm_d86b384b-923c-468d-9f67-6908e2065022/prometheus-operator/0.log" Jun 03 02:17:47.777684 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.777653 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-dxdzm_d86b384b-923c-468d-9f67-6908e2065022/kube-rbac-proxy/0.log" Jun 03 02:17:47.834552 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.834505 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-p9ffv_df01c07c-bdfa-4a36-948f-cffe32a31c79/prometheus-operator-admission-webhook/0.log" Jun 03 02:17:47.967031 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.966990 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85b6f6778d-9wpr8_79d0642a-f890-42dc-85e9-ca43ec3bbfcc/thanos-query/0.log" Jun 03 02:17:47.991294 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:47.991270 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85b6f6778d-9wpr8_79d0642a-f890-42dc-85e9-ca43ec3bbfcc/kube-rbac-proxy-web/0.log" Jun 03 02:17:48.018720 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:48.018681 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85b6f6778d-9wpr8_79d0642a-f890-42dc-85e9-ca43ec3bbfcc/kube-rbac-proxy/0.log" Jun 03 02:17:48.044736 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:48.044694 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85b6f6778d-9wpr8_79d0642a-f890-42dc-85e9-ca43ec3bbfcc/prom-label-proxy/0.log" Jun 03 02:17:48.068589 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:48.068511 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85b6f6778d-9wpr8_79d0642a-f890-42dc-85e9-ca43ec3bbfcc/kube-rbac-proxy-rules/0.log" Jun 03 02:17:48.092190 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:48.092154 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85b6f6778d-9wpr8_79d0642a-f890-42dc-85e9-ca43ec3bbfcc/kube-rbac-proxy-metrics/0.log" Jun 03 02:17:49.890856 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:49.890826 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/2.log" Jun 03 02:17:49.894999 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:49.894979 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-xxmhj_b16cd5f5-9216-4743-a5ab-97375ec370c1/console-operator/3.log" Jun 03 02:17:50.909769 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:50.909734 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-c49dr_fe9034a5-22c3-4644-9035-492d013fa9c0/volume-data-source-validator/0.log" Jun 03 02:17:51.702419 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.702387 2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz"] Jun 03 02:17:51.702862 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.702847 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6c51aa-1d60-4230-be2e-3d137358d9cc" containerName="manager" Jun 03 02:17:51.702914 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.702863 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6c51aa-1d60-4230-be2e-3d137358d9cc" containerName="manager" Jun 03 02:17:51.702963 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.702930 2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="6c6c51aa-1d60-4230-be2e-3d137358d9cc" containerName="manager" Jun 03 02:17:51.707312 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.707292 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.709970 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.709945 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x4cvb\"/\"openshift-service-ca.crt\"" Jun 03 02:17:51.709970 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.709963 2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x4cvb\"/\"default-dockercfg-fv2mg\"" Jun 03 02:17:51.711023 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.711005 2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x4cvb\"/\"kube-root-ca.crt\"" Jun 03 02:17:51.715863 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.715838 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz"] Jun 03 02:17:51.738730 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.738700 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bwd6s_74c31246-3616-41a4-b95a-0a5242f1fa28/dns/0.log" Jun 03 02:17:51.761564 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.761505 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bwd6s_74c31246-3616-41a4-b95a-0a5242f1fa28/kube-rbac-proxy/0.log" Jun 03 02:17:51.833656 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.833612 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-lib-modules\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.833656 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.833658 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-podres\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.833919 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.833750 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-sys\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.833919 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.833799 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-proc\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.833919 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.833820 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hb84\" (UniqueName: \"kubernetes.io/projected/4466d6e3-f78c-4726-8b14-88bbee31e8d5-kube-api-access-9hb84\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.907996 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.907961 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gv8d4_ff1bd7c9-7d88-4354-9d35-1fbfa1531cf4/dns-node-resolver/0.log" Jun 03 02:17:51.935324 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935282 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-sys\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935350 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-proc\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935374 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hb84\" (UniqueName: \"kubernetes.io/projected/4466d6e3-f78c-4726-8b14-88bbee31e8d5-kube-api-access-9hb84\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935404 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-sys\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935464 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-proc\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935482 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-lib-modules\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935516 2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-podres\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935614 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-lib-modules\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.935836 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.935660 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4466d6e3-f78c-4726-8b14-88bbee31e8d5-podres\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:51.943277 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:51.943249 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hb84\" (UniqueName: \"kubernetes.io/projected/4466d6e3-f78c-4726-8b14-88bbee31e8d5-kube-api-access-9hb84\") pod \"perf-node-gather-daemonset-9b2pz\" (UID: \"4466d6e3-f78c-4726-8b14-88bbee31e8d5\") " pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:52.018507 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.018406 2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:52.157998 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.157972 2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz"] Jun 03 02:17:52.161114 ip-10-0-136-201 kubenswrapper[2553]: W0603 02:17:52.161080 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4466d6e3_f78c_4726_8b14_88bbee31e8d5.slice/crio-4cf3ca4af90b51912d01e3f68fc8121f49fe04c07b8f9fd30c43a25a0b978ae8 WatchSource:0}: Error finding container 4cf3ca4af90b51912d01e3f68fc8121f49fe04c07b8f9fd30c43a25a0b978ae8: Status 404 returned error can't find the container with id 4cf3ca4af90b51912d01e3f68fc8121f49fe04c07b8f9fd30c43a25a0b978ae8 Jun 03 02:17:52.162840 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.162820 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 02:17:52.405567 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.405508 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" event={"ID":"4466d6e3-f78c-4726-8b14-88bbee31e8d5","Type":"ContainerStarted","Data":"ed71a1ea66cbfd75d071b3e26f5286cd5f78e4bf74b90376d1f0f6650bee3085"} Jun 03 02:17:52.405567 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.405560 2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" event={"ID":"4466d6e3-f78c-4726-8b14-88bbee31e8d5","Type":"ContainerStarted","Data":"4cf3ca4af90b51912d01e3f68fc8121f49fe04c07b8f9fd30c43a25a0b978ae8"} Jun 03 02:17:52.405773 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.405616 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:52.422430 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.422370 2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" podStartSLOduration=1.422353307 podStartE2EDuration="1.422353307s" podCreationTimestamp="2026-06-03 02:17:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 02:17:52.422061547 +0000 UTC m=+2243.076704828" watchObservedRunningTime="2026-06-03 02:17:52.422353307 +0000 UTC m=+2243.076996586" Jun 03 02:17:52.434563 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:52.433376 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t55ss_c3da5575-9d26-4064-85da-7c11be51c8d4/node-ca/0.log" Jun 03 02:17:53.236685 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:53.236649 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f8qhjnf_18f717c7-f71f-451f-8e35-68cf623ff703/istio-proxy/0.log" Jun 03 02:17:53.428801 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:53.428762 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-nlx4f_2f586b8f-dd7d-4ed3-abb2-d3d758b73bb0/istio-proxy/0.log" Jun 03 02:17:54.107759 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:54.107727 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x4r47_cc11711b-16fb-4148-bc98-81dbbc80b18d/serve-healthcheck-canary/0.log" Jun 03 02:17:54.674910 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:54.674879 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvhlg_57f1efd1-c82a-448a-b70a-081c94dc6f71/kube-rbac-proxy/0.log" Jun 03 02:17:54.694239 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:54.694196 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvhlg_57f1efd1-c82a-448a-b70a-081c94dc6f71/exporter/0.log" Jun 03 02:17:54.714851 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:54.714824 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvhlg_57f1efd1-c82a-448a-b70a-081c94dc6f71/extractor/0.log" Jun 03 02:17:56.805383 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:56.805351 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7b94c665b4-n5hkv_28135a58-5e1b-49ad-aa8d-09c8296676ce/nginx/0.log" Jun 03 02:17:57.342181 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:57.342145 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf_cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7/manager/0.log" Jun 03 02:17:57.359119 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:57.359095 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9cd4ffb9-fh2pf_cae7f9a0-bd6d-46d7-a8c7-87aafc235ed7/copy-manifests/0.log" Jun 03 02:17:58.421854 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:58.421821 2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x4cvb/perf-node-gather-daemonset-9b2pz" Jun 03 02:17:58.675245 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:17:58.675165 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bc87d8b78-q46fp_a798cf0d-3139-446f-91c0-2b8763a10266/manager/0.log" Jun 03 02:18:03.422468 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:03.422430 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-cx5mt_6ea6b0ba-a9a8-4710-a542-79a5aace89d3/migrator/0.log" Jun 03 02:18:03.442766 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:03.442733 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-cx5mt_6ea6b0ba-a9a8-4710-a542-79a5aace89d3/graceful-termination/0.log" Jun 03 02:18:03.773774 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:03.773741 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-tqsx7_8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0/kube-storage-version-migrator-operator/1.log" Jun 03 02:18:03.774892 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:03.774874 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-tqsx7_8ffbed46-5345-4dfc-8bf6-b2ee67fa59b0/kube-storage-version-migrator-operator/0.log" Jun 03 02:18:05.051760 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.051728 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/kube-multus-additional-cni-plugins/0.log" Jun 03 02:18:05.072224 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.072196 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/egress-router-binary-copy/0.log" Jun 03 02:18:05.093676 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.093648 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/cni-plugins/0.log" Jun 03 02:18:05.115855 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.115821 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/bond-cni-plugin/0.log" Jun 03 02:18:05.136584 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.136557 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/routeoverride-cni/0.log" Jun 03 02:18:05.156657 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.156625 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/whereabouts-cni-bincopy/0.log" Jun 03 02:18:05.177050 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.177016 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rd2nk_e1e6ff2f-3769-40c8-b42c-ba311fd1deca/whereabouts-cni/0.log" Jun 03 02:18:05.264277 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.264244 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r29r8_0ea6fc9a-0e75-4de7-9c7b-9b76ca3c226a/kube-multus/0.log" Jun 03 02:18:05.283938 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.283902 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-66gkf_b11762dd-14ed-48d3-a346-a3997ab3567b/network-metrics-daemon/0.log" Jun 03 02:18:05.304799 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:05.304724 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-66gkf_b11762dd-14ed-48d3-a346-a3997ab3567b/kube-rbac-proxy/0.log" Jun 03 02:18:06.175252 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.175218 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/ovn-controller/0.log" Jun 03 02:18:06.204892 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.204863 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/ovn-acl-logging/0.log" Jun 03 02:18:06.225032 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.225004 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/kube-rbac-proxy-node/0.log" Jun 03 02:18:06.247636 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.247600 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/kube-rbac-proxy-ovn-metrics/0.log" Jun 03 02:18:06.269710 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.269684 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/northd/0.log" Jun 03 02:18:06.289664 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.289639 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/nbdb/0.log" Jun 03 02:18:06.310186 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.310160 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/sbdb/0.log" Jun 03 02:18:06.419733 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:06.419702 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-475tz_b8ae6056-f1d4-4ec0-8d77-3981347bcbd5/ovnkube-controller/0.log" Jun 03 02:18:07.989252 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:07.989221 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56d84b9f44-x45hh_4476b4b9-86c6-4c55-95ef-ae4524693eaf/check-endpoints/0.log" Jun 03 02:18:08.063827 ip-10-0-136-201 kubenswrapper[2553]: I0603 02:18:08.063798 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-mv2xg_11abe95e-330a-4c61-8277-d49187327efc/network-check-target-container/0.log"